{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# LightGBM模型调优"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 首先 import 必要的模块\n",
    "import pandas as pd \n",
    "import numpy as np\n",
    "\n",
    "import lightgbm as lgbm\n",
    "from lightgbm.sklearn import LGBMClassifier\n",
    "\n",
    "from sklearn.model_selection import GridSearchCV\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 读取数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true,
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "# 读取数据\n",
    "train = pd.read_csv(\"FE_train.csv\")\n",
    "\n",
    "y_train = train['Disbursed'] \n",
    "X_train = train.drop([\"ID\", \"Disbursed\"], axis=1)\n",
    "\n",
    "#保存特征名字以备后用（可视化）\n",
    "feat_names = X_train.columns "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## LightGBM超参数调优"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "LightGBM的主要的超参包括：\n",
    "1. 树的数目n_estimators 和 学习率 learning_rate\n",
    "2. 树的最大深度max_depth 和 树的最大叶子节点数目num_leaves（LightGBM采用叶子优先的方式生成树，num_leaves很重要，设置成比 2^max_depth 小），通常设置为60～80\n",
    "3. 叶子结点的最小样本数:min_data_in_leaf(min_data, min_child_samples)\n",
    "4. 每棵树的列采样比例：feature_fraction/colsample_bytree\n",
    "5. 每棵树的行采样比例：bagging_fraction （需同时设置bagging_freq=1）/subsample\n",
    "6. 正则化参数lambda_l1(reg_alpha), lambda_l2(reg_lambda)\n",
    "\n",
    "7. 两个非模型复杂度参数，但会影响模型速度和精度。可根据特征取值范围和样本数目修改这两个参数\n",
    "1）特征的最大bin数目max_bin：默认255；\n",
    "2）用来建立直方图的样本数目subsample_for_bin：默认200000。\n",
    "\n",
    "对n_estimators，用LightGBM内嵌的cv函数调优，因为同XGBoost一样，LightGBM学习的过程内嵌了cv，速度极快。\n",
    "其他参数用GridSearchCV"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "MAX_ROUNDS = 10000"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 相同的交叉验证分组\n",
    "样本数太多（87020），CV折数越多，cv性能越好。\n",
    "可能是由于GBDT是很复杂的模型，其实数据越多越好（cv折数多，每次留出的样本少，参数模型训练的样本更多）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# prepare cross validation\n",
    "from sklearn.model_selection import StratifiedKFold\n",
    "\n",
    "kfold = StratifiedKFold(n_splits=5, shuffle=True, random_state=3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index([u'City', u'Device_Type', u'EMI_Loan_Submitted', u'Employer_Name',\n",
       "       u'Existing_EMI', u'Filled_Form', u'Gender', u'Interest_Rate',\n",
       "       u'Loan_Amount_Applied', u'Loan_Amount_Submitted',\n",
       "       u'Loan_Tenure_Applied', u'Loan_Tenure_Submitted', u'Mobile_Verified',\n",
       "       u'Monthly_Income', u'Processing_Fee', u'Salary_Account', u'Source',\n",
       "       u'Var1', u'Var2', u'Var4', u'Var5', u'Age'],\n",
       "      dtype='object')"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train.columns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1. n_estimators"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#直接调用lightgbm内嵌的交叉验证(cv)，可对连续的n_estimators参数进行快速交叉验证\n",
    "#而GridSearchCV只能对有限个参数进行交叉验证，且速度相对较慢\n",
    "def get_n_estimators(params , X_train , y_train , early_stopping_rounds=10):\n",
    "    lgbm_params = params.copy()\n",
    "     \n",
    "    lgbmtrain = lgbm.Dataset(X_train , y_train )\n",
    "     \n",
    "    #num_boost_round为弱分类器数目，下面的代码参数里因为已经设置了early_stopping_rounds\n",
    "    #即性能未提升的次数超过过早停止设置的数值，则停止训练\n",
    "    cv_result = lgbm.cv(lgbm_params , lgbmtrain , num_boost_round=MAX_ROUNDS , nfold=5,  metrics='auc' , early_stopping_rounds=early_stopping_rounds,seed=3 )\n",
    "     \n",
    "    print('best n_estimators:' , len(cv_result['auc-mean']))\n",
    "    print('best cv score:' , cv_result['auc-mean'][-1])\n",
    "     \n",
    "    return len(cv_result['auc-mean'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "('best n_estimators:', 38)\n",
      "('best cv score:', 0.8455091020436194)\n"
     ]
    }
   ],
   "source": [
    "params = {'boosting_type': 'goss',\n",
    "          'objective': 'binary',\n",
    "          'is_unbalance':True,\n",
    "          #'categorical_feature': names:'City', 'Employer_Name', 'Salary_Account','Device_Type','Filled_Form','Gender','Mobile_Verified','Source','Var1','Var2','Var4',\n",
    "          'categorical_feature': [0,1,3,5,6,12,15,16,17,18,19,20],\n",
    "          'n_jobs': 4,\n",
    "          'learning_rate': 0.1,\n",
    "          #'n_estimators':n_estimators_1,\n",
    "          'num_leaves': 60,\n",
    "          'max_depth': 6,\n",
    "          'colsample_bytree': 0.7,\n",
    "          'verbosity':5\n",
    "         }\n",
    "\n",
    "#categorical_feature = ['City', 'Employer_Name', 'Salary_Account','Device_Type','Filled_Form','Gender','Mobile_Verified','Source','Var1','Var2','Var4']\n",
    "n_estimators_1 = get_n_estimators(params, X_train , y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2. num_leaves & max_depth=7\n",
    "num_leaves建议70-80，搜索区间50-80,值越大模型越复杂，越容易过拟合\n",
    "相应的扩大max_depth=7"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fitting 5 folds for each of 4 candidates, totalling 20 fits\n",
      "[CV] num_leaves=50 ...................................................\n",
      "[CV] num_leaves=50 ...................................................\n",
      "[CV] num_leaves=50 ...................................................\n",
      "[CV] num_leaves=50 ...................................................\n",
      "[CV] .............. num_leaves=50, score=0.833265759508, total=   0.8s\n",
      "[CV] ............... num_leaves=50, score=0.86334968273, total=   0.9s\n",
      "[CV] num_leaves=50 ...................................................\n",
      "[CV] .............. num_leaves=50, score=0.843559092769, total=   0.8s\n",
      "[CV] .............. num_leaves=50, score=0.832192191162, total=   0.9s\n",
      "[CV] num_leaves=60 ...................................................\n",
      "[CV] num_leaves=60 ...................................................\n",
      "[CV] num_leaves=60 ...................................................\n",
      "[CV] .............. num_leaves=50, score=0.846767309955, total=   0.9s\n",
      "[CV] num_leaves=60 ...................................................\n",
      "[CV] .............. num_leaves=60, score=0.861492025382, total=   0.9s\n",
      "[CV] num_leaves=60 ...................................................\n",
      "[CV] .............. num_leaves=60, score=0.835143877366, total=   0.9s\n",
      "[CV] num_leaves=70 ...................................................\n",
      "[CV] .............. num_leaves=60, score=0.832997084548, total=   1.1s\n",
      "[CV] num_leaves=70 ...................................................\n",
      "[CV] .............. num_leaves=60, score=0.845292854706, total=   0.9s\n",
      "[CV] .............. num_leaves=60, score=0.846687761689, total=   0.9s\n",
      "[CV] num_leaves=70 ...................................................\n",
      "[CV] num_leaves=70 ...................................................\n",
      "[CV] .............. num_leaves=70, score=0.859989481507, total=   0.9s\n",
      "[CV] num_leaves=70 ...................................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=4)]: Done  10 tasks      | elapsed:    3.9s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV] .............. num_leaves=70, score=0.834432287201, total=   1.1s\n",
      "[CV] num_leaves=80 ...................................................\n",
      "[CV] .............. num_leaves=70, score=0.848371475943, total=   0.9s\n",
      "[CV] .............. num_leaves=70, score=0.836587624729, total=   1.0s\n",
      "[CV] num_leaves=80 ...................................................\n",
      "[CV] .............. num_leaves=70, score=0.846724953086, total=   0.9s\n",
      "[CV] num_leaves=80 ...................................................\n",
      "[CV] num_leaves=80 ...................................................\n",
      "[CV] .............. num_leaves=80, score=0.859989481507, total=   1.0s\n",
      "[CV] num_leaves=80 ...................................................\n",
      "[CV] .............. num_leaves=80, score=0.836587624729, total=   0.9s\n",
      "[CV] .............. num_leaves=80, score=0.848371475943, total=   0.9s\n",
      "[CV] .............. num_leaves=80, score=0.834432287201, total=   1.0s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=4)]: Done  18 out of  20 | elapsed:    6.5s remaining:    0.7s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV] .............. num_leaves=80, score=0.846724953086, total=   1.0s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=4)]: Done  20 out of  20 | elapsed:    6.7s finished\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "GridSearchCV(cv=StratifiedKFold(n_splits=5, random_state=3, shuffle=True),\n",
       "       error_score='raise',\n",
       "       estimator=LGBMClassifier(boosting_type='goss',\n",
       "        categorical_feature=[0, 1, 3, 5, 6, 12, 15, 16, 17, 18, 19, 20],\n",
       "        colsample_bytree=0.7, is_unbalance=True, learning_rate=0.1,\n",
       "        max_bin=255, max_depth=6, min_child_samples=10, min_child_weight=5,\n",
       "        min_split_gain=0.0, n_estimators=38, n_jobs=4, num_leaves=31,\n",
       "        objective='binary', random_state=0, reg_alpha=0.0, reg_lambda=0.0,\n",
       "        silent=False, subsample=1.0, subsample_for_bin=50000,\n",
       "        subsample_freq=1),\n",
       "       fit_params=None, iid=True, n_jobs=4,\n",
       "       param_grid={'num_leaves': [50, 60, 70, 80]},\n",
       "       pre_dispatch='2*n_jobs', refit=False, return_train_score='warn',\n",
       "       scoring='roc_auc', verbose=5)"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "params = {'boosting_type': 'goss',\n",
    "          'objective': 'binary',\n",
    "          'is_unbalance':True,\n",
    "          'categorical_feature': [0,1,3,5,6,12,15,16,17,18,19,20],\n",
    "          'n_jobs': 4,\n",
    "          'learning_rate': 0.1,\n",
    "          'n_estimators':n_estimators_1,\n",
    "          #'num_leaves': 60,\n",
    "          'max_depth': 7,\n",
    "          'colsample_bytree': 0.7,\n",
    "         }\n",
    "lg = LGBMClassifier(silent=False,  **params)\n",
    "\n",
    "num_leaves_s = range(50,90,10) #50,60,70,80\n",
    "tuned_parameters = dict( num_leaves = num_leaves_s)\n",
    "\n",
    "grid_search = GridSearchCV(lg, n_jobs=4, param_grid=tuned_parameters, cv = kfold, scoring=\"roc_auc\", verbose=5, refit = False)\n",
    "grid_search.fit(X_train , y_train)\n",
    "#grid_search.best_estimator_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.84522115674\n",
      "{'num_leaves': 70}\n"
     ]
    }
   ],
   "source": [
    "# examine the best model\n",
    "print(grid_search.best_score_)\n",
    "print(grid_search.best_params_)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZgAAAELCAYAAADkyZC4AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3Xl4VdW5x/HvD5BRBhFEZRAUVHBg\nSsGhWhWtiFZo64Ria4tFvWqdsGhrb8HWDtqKVlGL1uLFAVC0otWiba1Di0Mi8ygiQkQhoAyRMeS9\nf5xNG2MgAXI4Ocnv8zw+yV57nX3e5eacN2utvfZWRGBmZlbZamU6ADMzq56cYMzMLC2cYMzMLC2c\nYMzMLC2cYMzMLC2cYMzMLC2cYMzMLC2cYMzMLC2cYMzMLC3qZDqATGrRokW0b98+02GYmWWVvLy8\nlRHRsrx6NTrBtG/fntzc3EyHYWaWVSR9WJF6HiIzM7O0cIIxM7O0cIIxM7O0cIIxM7O0cIIxM7O0\ncIIxM7O0cIIxM7O0qNHrYMwsvT5c9TmvLSjIdBhWhlM6t6J1swZpfY+0JhhJfYG7gdrAQxHx61L7\n2wGPAM2SOjdFxAul9s8BhkfEb5OyxcA6YCtQFBE5SfkdwDeAzcD7wPciYnU622dm2/fxmg18+/5/\ns7Jwc6ZDsTK0bd4wexOMpNrAKOA0IB94R9KkiJhTototwISIuF9SF+AFoH2J/SOBF8s4/MkRsbJU\n2cvAzRFRJOk3wM3AsMppjZntjE1FW7n80XfZsHkrz155PK33Se8Xme28xvXTP4CVznfoBSyMiEUA\nksYB/Un1SLYJoEnye1Ng2bYdkgYAi4DPK/JmEfFSic03gXN2OXIz22URwf/+eTbTl67mgUE96dq2\nWaZDsgxJ5yR/a2Bpie38pKyk4cAgSfmkei9XA0hqRKr3MaKM4wbwkqQ8SUO2897fp+yeD5KGSMqV\nlFtQ4LFhs8r2+NtLGJ+7lKtP6UjfI/fPdDiWQelMMCqjLEptDwTGREQboB8wVlItUollZEQUlnGM\n4yOiB3AGcKWkE7/wptJPgCLgsbKCiojREZETETktW5Z7M1Az2wl5H37G8EmzOfmwllx76qGZDscy\nLJ1DZPlA2xLbbSgxBJYYDPQFiIgpkuoDLYDewDmSbid1AUCxpI0RcW9ELEvqr5D0DKmhuNcAJH0X\nOAvoExGlk5mZpdGKtRu54tE8DmzWgLvO707tWmX9jWk1STp7MO8AnSR1kFQXuACYVKrOEqAPgKTO\nQH2gICJOiIj2EdEeuAv4ZUTcK6mRpMZJ/UbA14FZyXZfUsNqZ0fE+jS2y8xK2VxUzBWPvUvhpiJG\nX5xD04Z7ZTokqwLS1oNJrua6CphM6hLkhyNitqRbgdyImATcADwo6TpSw2eXlNPzaAU8I2lb7I9H\nxF+TffcC9YCXk/1vRsTl6WibmX3Rrc/PJu/Dzxh1YQ8O279xpsOxKkI1eSQpJycn/MAxs90z/p0l\nDJs4k8u/dgg3nXF4psOxPUBS3rY1iDviW8WY2S6btnQ1P/3zbE7o1IIbTz8s0+FYFeMEY2a7pGDd\nJi4fm0erpvW4Z6An9e3LfC8yM9tpW7YWc+Vj77J6w2aevuJ4mjWsm+mQrApygjGznXbbX+by9uJP\nufuCbnQ5sEn5L7AayUNkZrZTnsrLZ8y/F/ODEzrQv1vpm3OY/ZcTjJlV2Mz8Nfz4mZkcd8i+DOvr\nK8Zsx5xgzKxCVhVu4rKxubTcux73XtiDOrX99WE75jkYMytX0dZirnp8Kqs+38zEK46jeSNP6lv5\nnGDMrFy/fnEeUxat4s7zunJk66aZDseyhPu4ZrZDz077iIfe+IBLjmvPt3q0yXQ4lkWcYMxsu2Yv\nW8OwiTPo1aE5Pzmzc6bDsSzjBGNmZfrs881cNjaPfRrWZdSFPdjLk/q2kzwHY2ZfUrS1mKufmMqK\ndZt48rJjadm4XqZDsizkP0nM7EvueGk+byxcyS8GHEnXts0yHY5lKScYM/uC52cs4w+vLuLiYw7i\nvJy25b/AbDucYMzsP+Z9spYbn5xBzkH78NOzumQ6HMtyTjBmBsDq9ZsZ8n95NGlQh/sG9aBuHX89\n2O7xJL+ZsbU4uGbcND5es4FxQ45lv8b1Mx2SVQNOMGbGnS/P59UFBfzym0fR86B9Mh2OVRNp7QNL\n6itpvqSFkm4qY387Sa9ImipphqR+ZewvlDS0RNliSTMlTZOUW6K8uaSXJb2X/PSnxKwC/jrrY0a9\n8j4De7Xlwt7tMh2OVSNpSzCSagOjgDOALsBASaVnDW8BJkREd+AC4L5S+0cCL5Zx+JMjoltE5JQo\nuwn4e0R0Av6ebJvZDry3fB03TJhOt7bNGH72EZkOx6qZdPZgegELI2JRRGwGxgH9S9UJYNvj8JoC\ny7btkDQAWATMruD79QceSX5/BBiwi3Gb1QhrN25hyNg8GtStwwODelKvTu1Mh2TVTDoTTGtgaYnt\n/KSspOHAIEn5wAvA1QCSGgHDgBFlHDeAlyTlSRpSorxVRHwMkPzcrzIaYVYdFRcH142bxtJP13P/\noB7s39ST+lb50plgVEZZlNoeCIyJiDZAP2CspFqkEsvIiCgs4xjHR0QPUkNvV0o6caeCkoZIypWU\nW1BQsDMvNas27v77e/x93gp+9o0ufKV980yHY9VUOhNMPlByGXAbSgyBJQYDEwAiYgpQH2gB9AZu\nl7QYuBb4saSrknrLkp8rgGdIDcUBLJd0AEDyc0VZQUXE6IjIiYicli1b7m4bzbLOy3OWc/ff3+Pc\nnm0YdMxBmQ7HqrF0Jph3gE6SOkiqS2oSf1KpOkuAPgCSOpNKMAURcUJEtI+I9sBdwC8j4l5JjSQ1\nTuo3Ar4OzEqONQn4bvL7d4Fn09c0s+y0cEUh142fxtFtmvLzAUcilTXQYFY50rYOJiKKkl7HZKA2\n8HBEzJZ0K5AbEZOAG4AHJV1HavjskogoPYxWUivgmeRDUQd4PCL+muz7NTBB0mBSievctDTMLEut\n27iFy8bmUq9OLR4Y1JP6e3lS39JLO/4+r95ycnIiNze3/IpmWa64OLj80Tz+Pm8Fj13am2MO3jfT\nIVkWk5RXaplImXyzIbMaYNQrC3lpznJ+0q+zk4vtMU4wZtXcP+Yt586/LeCb3VvzvePbZzocq0Gc\nYMyqsQ9Wfs4146bR5YAm/PKbR3lS3/YoJxizaurzTUUM+b9c6tQSDwzqSYO6ntS3Pct3UzarhiKC\nG5+azvsFhYwd3Ju2zRtmOiSrgdyDMauGHnh1ES/M/ISbz+jM8R1bZDocq6GcYMyqmVcXFHD75Hl8\no+uBXHpCh0yHYzWYE4xZNbJk1Xp++MRUDmvVmN9825P6lllOMGbVxPrNRQwZm1o4PPriHBrW9RSr\nZZYTjFk1EBEMmziTBcvXcc/A7rTb15P6lnlOMGbVwEOvf8Bz05cx9PTDOPFQ3yXcqgYnGLMs98Z7\nK/nVi3Ppd9T+XPG1QzIdjtl/OMGYZbGln67n6ifepeN+e3PHOV09qW9VihOMWZbasHkrl43No6g4\n+MPFOTSq50l9q1r8L9IsC0UENz89g7mfrOWP382hQ4tGmQ7J7EvcgzHLQn/612L+PG0Z1596KKcc\n3irT4ZiVyQnGLMu8uWgVt70wl693acWVJ3fMdDhm2+UEY5ZFlq3ewJWPvUv7fRvyu/O6UquWJ/Wt\n6nKCMcsSG7ds5fJH89hUVMzo7+TQuP5emQ7JbIfSmmAk9ZU0X9JCSTeVsb+dpFckTZU0Q1K/MvYX\nShpaqrx28prnS5T1kfSupGmS3pDksQOrNiKCW/48ixn5axh5fjcOabl3pkMyK1faEoyk2sAo4Ayg\nCzBQUpdS1W4BJkREd+AC4L5S+0cCL5Zx+GuAuaXK7gcuiohuwOPJsc2qhUff/JCn8vL5YZ9OnNbF\nk/qWHdLZg+kFLIyIRRGxGRgH9C9VJ4Amye9NgWXbdkgaACwCZpd8gaQ2wJnAQxU9llk2e2fxp4x4\nbg59Dt+Pa/t0ynQ4ZhWWznUwrYGlJbbzgd6l6gwHXpJ0NdAIOBVAUiNgGHAaMLTUa+4CfgQ0LlV+\nKfCCpA3AWuCY3W+CWWZ9smYjVzz6Lm2bN+TO87t5Ut+ySjp7MGV9EqLU9kBgTES0AfoBYyXVAkYA\nIyOi8AsHlM4CVkREXhnHvg7olxzrT8CdZQYlDZGUKym3oKBg51pktgdtKkpN6m/YXMToi3vStIEn\n9S27pLMHkw+0LbHdhi8PWw0G+gJExBRJ9YEWpHo650i6HWgGFEvaSKpXdHZyMUB9oImkR0kll64R\n8VZy3PHAX8sKKiJGA6MBcnJySic8sypj+KTZTFu6mgcG9aBTq9IddrOqL50J5h2gk6QOwEekJvEv\nLFVnCdAHGCOpM6mkURARJ2yrIGk4UBgR9yZFNyflJwFDI2KQpDpAU0mHRsQCUkNrpS8CMMsaj7+1\nhCfeXsqVJx9C3yMPyHQ4ZrskbQkmIookXQVMBmoDD0fEbEm3ArkRMQm4AXhQ0nWkhs8uiYid7lUk\n7/UDYKKkYuAz4PuV1hizPSjvw8/42aRZfO3Qllx/2mGZDsdsl2kXvs+rjZycnMjNzc10GGb/sWLt\nRs665w3q71Wb5676Kk0bet7Fqh5JeRGRU149r+Q3qyI2FxXzP4+9y7qNRYz+Tk8nF8t6vl2/WRXx\n8+fnkPvhZ9x7YXcO379J+S8wq+LcgzGrAia8s5Sxb37IZScezFlHH5jpcMwqhROMWYZNW7qaW/48\ni692bMGNp3tS36oPJxizDCpYt4nLx+axX5N63DOwO3Vq+yNp1YfnYMwyZMvWYq58/F1Wb9jMxCuO\nY59GdTMdklmlcoIxy5Db/jKXtz/4lLsv6MYRBzbNdDhmlc79cbMMmJiXz5h/L2bwVzvQv1vrTIdj\nlhZOMGZ72Mz8Nfz4mZkcc3Bzbj7j8EyHY5Y2TjBme9Cqwk1c/mge+zaqy6gLe3hS36o1z8GY7SFF\nW4u56vGpFBRuYuLlx7Hv3vUyHZJZWvnPJ7M95NcvzmPKolX86ptHcVQbT+pb9ecEY7YHPDvtIx56\n4wMuOa493+7ZJtPhmO0RTjBmaTZn2VqGTZxBr/bN+cmZnTMdjtke4wRjlkaffb6Zyx7NpVmDuoy6\nqAd7eVLfahBP8pulydbi4IfjprJ8zSbGX3YMLRt7Ut9qFicYszS5Y/J8Xn9vJb/59lF0b7dPpsMx\n2+PcXzdLg+dnLOOBV9/not7tOP8r7TIdjllGOMGYVbJ5n6zlxidn0POgffjZN47IdDhmGeMEY1aJ\n1qzfwmVj82hcvw73X9SDunX8EbOaK63/+iX1lTRf0kJJN5Wxv52kVyRNlTRDUr8y9hdKGlqqvHby\nmudLlEnSbZIWSJor6Yfpa5nZl20tDq4ZP5Vlqzdw/6Ae7NekfqZDMsuotE3yS6oNjAJOA/KBdyRN\niog5JardAkyIiPsldQFeANqX2D8SeLGMw18DzAVKPrj8EqAtcHhEFEvar7LaYlYRI19ewD/nF3Db\nN4+k50HNMx2OWcalswfTC1gYEYsiYjMwDuhfqk7w3yTRFFi2bYekAcAiYHbJF0hqA5wJPFTqWFcA\nt0ZEMUBErKikdpiV66+zPubeVxZywVfacmEvT+qbQXoTTGtgaYnt/KSspOHAIEn5pHovVwNIagQM\nA0aUcdy7gB8BxaXKDwHOl5Qr6UVJncoKStKQpE5uQUHBTjbJ7MveW76OGyZMp2vbZozofwSSMh2S\nWZWQzgRT1qcsSm0PBMZERBugHzBWUi1SiWVkRBR+4YDSWcCKiMgr49j1gI0RkQM8CDxcVlARMToi\nciIip2XLljvXIrNS1m7cwpCxeTSoW5sHBvWgXp3amQ7JrMpI50LLfFJzItu0ocQQWGIw0BcgIqZI\nqg+0AHoD50i6HWgGFEvaSKoHdHZyMUB9oImkRyNiUPJ+E5PjPgP8KT3NMkspLg6uHz+NpZ+u57FL\ne3NA0waZDsmsStluD0bS6ZLOKaP8IkmnVeDY7wCdJHWQVBe4AJhUqs4SoE9y3M6kkkZBRJwQEe0j\noj2pIbFfRsS9EXFzRLRJyi8A/pEkF4A/A6ckv38NWFCBGM122e//8R5/m7uCn57Vhd4H75vpcMyq\nnB0NkY0AXi2j/O/AreUdOCKKgKuAyaSu+JoQEbMl3Srp7KTaDcAPJE0HngAuiYjSw2gV9Wvg25Jm\nAr8CLt3F45iV6+U5y7nrb+/x7R5t+M6xB2U6HLMqSdv7Ppc0IyKO3tl92SQnJydyc3MzHYZlmfcL\nChlw779o36IRT15+LPX38ryL1SyS8pL57h3aUQ+mvqQvzdFI2gvwYLPVSOs2bmHI/+WyV51aPHBx\nTycXsx3YUYJ5GngwuWQY+M/lww8k+8xqlOLi4IYJ01m8aj33Xtid1s38d5bZjuwowdwCLAc+lJQn\n6V1gMVCQ7DOrUe7750JemrOcH/frzHGHtMh0OGZV3nYvU04m6W+SNALomBQvjIgNeyQysyrklXkr\n+N3LCxjQ7UC+f3z7TIdjlhW2m2AkfatUUQDNJE2LiHXpDcus6li88nN+OG4qnfdvwq++dbRX6ptV\n0I4WWn6jjLLmwNGSBkfEP9IUk1mV8fmmIoaMzaV2LfGHi3vSoK4n9c0qakdDZN8rq1zSQcAEUqvt\nzaqtiODGp6azcEUhj3y/F22bN8x0SGZZZafvRRYRHwJ7pSEWsyrlgVcX8cLMTxjW93BO6OT71pnt\nrJ1OMJIOBzalIRazKuO1BQXcMXkeZx59AENOPDjT4ZhlpR1N8j/Hl+9+3Bw4ABj05VeYVQ9LVq3n\n6iemcmirxtxxjif1zXbVjib5f1tqO4BPSSWZQcCUdAVllinrN6cm9SOCP1zck4Z103nDcbPqbUeT\n/P+50aWkbsCFwHnAB/z3tvhm1UZEcNPEmcxfvo4/XfIVDtq3UfkvMrPt2tEQ2aGkbok/EFgFjCd1\nc8yT91BsZnvUH9/4gEnTl3Hj6Ydx0mH7ZTocs6y3o/7/POB14BsRsRBA0nV7JCqzPezfC1fyyxfm\ncsaR+/M/Jx2S6XDMqoUdXUX2beAT4BVJD0rqQ9mPQTbLavmfrefKx9/lkJZ7c8e5XT2pb1ZJtptg\nIuKZiDgfOBz4J3Ad0ErS/ZK+vofiM0urjVu2ctnYPIq2pib1967nSX2zylLuOpiI+DwiHouIs4A2\nwDTgprRHZpZmEcHNT89k9rK13HVBNw5uuXemQzKrVnZqoWVEfBoRf4iIU9IVkNmeMubfi3lm6kdc\nd+qh9OncKtPhmFU7O72S36w6eHPRKn7xl7mc2rkVV5/SsfwXmNlOS2uCkdRX0nxJCyV9aVhNUjtJ\nr0iaKmmGpH5l7C+UNLRUee3kNc+Xccx7JBVWfmusuli2egNXPvYuB+3bkDvP70qtWp7UN0uHtCUY\nSbWBUcAZQBdgoKQupardAkyIiO6k1tzcV2r/SODFMg5/DTC3jPfMAZrtZuhWjW3cspUrHs1jU1Ex\noy/OoUl937fVLF3S2YPpReoJmIsiYjMwDuhfqk4ATZLfmwLLtu2QNABYBMwu+QJJbYAzgYdKldcG\n7gB+VIltsGokIvjpn2cxPX8NvzuvKx3386S+WTqlM8G0BpaW2M5PykoaDgySlA+8AFwNIKkRMAwY\nUcZx7yKVRIpLlV8FTIqIj3c7cquWHn1rCU/m5fPDUzpy+hH7Zzocs2ovnQmmrIHt0ndnHgiMiYg2\nQD9grKRapBLLyIj4wlyKpLOAFRGRV6r8QOBc4J5yg5KGSMqVlFtQUFDx1lhWe2fxp4yYNJuTD2vJ\ntacemulwzGqEdK4qywfalthuQ4khsMRgoC9AREyRVB9oQeppmedIup3UnEqxpI2kekBnJxcD1Aea\nSHoUeALoCCxMVmE3lLQwIr50eVBEjAZGA+Tk5JROeFYNfbJmI1c8+i5t9mnAXRd096S+2R6SzgTz\nDtBJUgfgI1KT+BeWqrME6AOMkdSZVNIoiIgTtlWQNBwojIh7k6Kbk/KTgKERse3ZNPuXeE1hWcnF\nap5NRVu54rE81m8u4vEf9KZpA0/qm+0paRsii4giUvMik0ld8TUhImZLulXS2Um1G4AfSJpOqhdy\nSUS4V2GVZvikOUxdsprfntuVQ1s1znQ4ZjWKavL3eU5OTuTm5mY6DEuTx99awo+fmckVJx3CsL6H\nZzocs2pDUl5E5JRXzyv5rVrK+/AzfjZpFid0asHQrx+W6XDMaiQnGKt2VqzdyBWP5rF/0/rcM7A7\ntT2pb5YRTjBWrWwuKuZ/HnuXdRuLGH1xDs0a1s10SGY1lh9+YdXKz5+fQ+6Hn/H7gd3pfECT8l9g\nZmnjHoxVGxNylzL2zQ8ZcuLBnN31wEyHY1bjuQdjWa+4OBjz78X8+sV5HN9xX350uif1zaoCJxjL\nastWb+DGp6bzr4WrOOXw/bjzvK7Uqe2OuVlV4ARjWSkieHbaMn767Cy2Fge//OZRDOzVluRWQWZW\nBTjBWNZZvX4zP/nzLP4y42N6tGvGyPO7cdC+jTIdlpmV4gRjWeXVBQXc+OR0Pv18MzeefhiXnXiw\nh8TMqignGMsK6zcX8asX5jH2zQ/ptN/ePHzJVziyddNMh2VmO+AEY1Xe1CWfcf2E6Xyw8nMGf7UD\nN55+GPX3qp3psMysHE4wVmVt2VrMPf9YyKhXFtKqcT0ev7Q3x3VskemwzKyCnGCsSlq4opDrJ0xj\nRv4avtW9NT87+wg/y8UsyzjBWJVSXBz835TF/OrFeTSoW5v7LupBv6MOyHRYZrYLnGCsyvhkzUZu\nfGo6r7+3kpMOa8nt3z6a/ZrUz3RYZraLnGCsSpg0fRm3PDOTLVuDXww4kot6t/OiSbMs5wRjGbV6\n/WZ++uxsnpu+jG5tU4smO7Twokmz6sAJxjLm9fcKGPrkdFYVbuaG0w7lipMO8aJJs2okrZ9mSX0l\nzZe0UNJNZexvJ+kVSVMlzZDUr4z9hZKGliqvnbzm+RJljyXvNUvSw5J8yVEVtWHzVn727Cwu/uPb\n7F2vDs/8z/Fc3aeTk4tZNZO2T7Sk2sAo4AygCzBQUpdS1W4BJkREd+AC4L5S+0cCL5Zx+GuAuaXK\nHgMOB44CGgCX7lYDLC2mL13Nmfe8ziNTPuR7x7fnLz88gaPaeEW+WXWUziGyXsDCiFgEIGkc0B+Y\nU6JOANseO9gUWLZth6QBwCLg85IHldQGOBO4Dbj+PweKeKFEnbeBNpXYFttNW7YWM+qVhdzzj4Xs\n17gejw7uzVc7edGkWXWWzgTTGlhaYjsf6F2qznDgJUlXA42AUwEkNQKGAacBQ0u95i7gR0Djst40\nGRq7mFQvx6qARQWFXDdhOtOXrmZAtwMZ0f9IL5o0qwHSOehd1jWmUWp7IDAmItoA/YCxkmoBI4CR\nEVH4hQNKZwErIiJvB+97H/BaRLxeZlDSEEm5knILCgoq2hbbBRHB2CmL6ff711m88nPuvbA7d13Q\n3cnFrIZIZw8mH2hbYrsNJYbAEoOBvgARMUVSfaAFqZ7OOZJuB5oBxZI2kuoVnZ1cDFAfaCLp0YgY\nBCDpZ0BL4LLtBRURo4HRADk5OaUTnlWS5Ws3cuNTM3htQQEnHppaNLl/Uy+aNKtJ0plg3gE6SeoA\nfERqEv/CUnWWAH2AMZI6k0oaBRFxwrYKkoYDhRFxb1J0c1J+EjC0RHK5FDgd6BMRxelqlJXv+RnL\n+Mkzs9hUtJWf9z+CQccc5EWTZjVQ2hJMRBRJugqYDNQGHo6I2ZJuBXIjYhJwA/CgpOtIDZ9dEhG7\n2qt4APgQmJJ8mT0dEbfudkOswtas38L/TprFs9OW0bVtM+48ryuHtNw702GZWYZo17/Ps19OTk7k\n5uZmOoxq4Y33VjL0yekUFG7ih6d04sqTvWjSrLqSlBcROeXV80p+2y0bt2zl1y/OY8y/F3Nwy0Y8\nffFxdG3bLNNhmVkV4ARju2xm/hquHT+V9ws+55Lj2jOs7+E0qOsnTZpZihOM7bSircXc/8/3ufvv\n77Hv3nUZO7gXJ3RqmemwzKyKcYKxnfLBys+5fsI0pi5ZzTe6HsjP+x9Bs4Z1Mx2WmVVBTjBWIRHB\nY28t4ba/zGWv2uL3A7tzdtcDMx2WmVVhTjBWrhVrN/KjiTP45/wCTujUgtvPOZoDmjbIdFhmVsU5\nwdgOvTDzY378zEw2bN7KiLOP4OJjDqJWLS+aNLPyOcFYmdZs2MLwSbN5ZupHHN2mKXee142O+3nR\npJlVnBOMfcm/F6YWTS5ft4lr+nTiqlM6spcXTZrZTnKCsf/YuGUrd0yezx/f+ICDWzRi4hXH0c2L\nJs1sFznBGACzPlrDdeOn8d6KQr5z7EHcfEZnL5o0s93iBFPDFW0t5g+vLWLkywto3qguj3y/F187\n1IsmzWz3OcHUYIuTRZPvLlnNmUcfwC/6H8k+jbxo0swqhxNMDRQRPPH2Un7xlznUriXuvqAbZ3c9\n0M9sMbNK5QRTw6xYt5GbJs7kH/NWcHzHfbnjnK4c2MyLJs2s8jnB1CB/nfUxNz89k/Wbt/Kzb3Th\nu8e296JJM0sbJ5gaYO3GLYyYNIeJ7+ZzVOumjDy/Kx33a5zpsMysmnOCqeamvL+KoU9O5+M1G/jh\nKR25uk8nL5o0sz3CCaaa2rhlK797aT4PvfEBBzVvyFNXHEePdvtkOiwzq0GcYKqh2cvWcP346cxf\nvo6LerfjJ2d2pmFdn2oz27PSOlYiqa+k+ZIWSrqpjP3tJL0iaaqkGZL6lbG/UNLQUuW1k9c8X6Ks\ng6S3JL0nabykGregY2txcN8/FzJg1L/4dP1m/vS9r3DbN49ycjGzjEhbgpFUGxgFnAF0AQZK6lKq\n2i3AhIjoDlwA3Fdq/0jgxTIOfw0wt1TZb4CREdEJ+AwYvHstyC5LVq3n/D9M4fa/zue0Lq2YfO2J\nnHzYfpkOy8xqsHT2YHoBCyNiUURsBsYB/UvVCaBJ8ntTYNm2HZIGAIuA2SVfIKkNcCbwUIkyAacA\nTyVFjwADKq0lVVhEMO7tJfS9+zXmf7KOked3ZdSFPWjuFflmlmHpHDtpDSwtsZ0P9C5VZzjwkqSr\ngUbAqQCSGgHDgNOAoaVecxcNvQY+AAAMmUlEQVTwI6Dkdbb7AqsjoqjEe7UuKyhJQ4AhAO3atdup\nBlU1Bes2cfPTM/jb3BUce/C+/Pa8rrT2okkzqyLS2YMpawVflNoeCIyJiDZAP2CspFrACFLDXYVf\nOKB0FrAiIvJ24b1ShRGjIyInInJatszemzpOnv0Jp9/1Gq+9t5KfntWFxy7t7eRiZlVKOnsw+UDb\nEtttKDEElhgM9AWIiCmS6gMtSPV0zpF0O9AMKJa0kVSv5OzkYoD6QBNJjwIXA80k1Ul6MWW9V7Ww\nbuMWbn1uDk/m5XPEgU0YeX43Dm3lRZNmVvWkM8G8A3SS1AH4iNQk/oWl6iwB+gBjJHUmlTQKIuKE\nbRUkDQcKI+LepOjmpPwkYGhEDEq2XwHOITXX813g2fQ0K3PeWrSKG56czrLVG7jy5EO4ps+h1K3j\nRZNmVjWlLcFERJGkq4DJQG3g4YiYLelWIDciJgE3AA9Kuo7UkNYlEVHm0FYFDAPGSfoFMBX44+63\nomrYVLSVO19awOjXF9GueUOevPxYeh7UPNNhmZntkHb9+zz75eTkRG5ubqbD2KG5H6/luvHTmPfJ\nOgb2asctZ3amUT2vazGzzJGUFxE55dXzN1UVtbU4ePD1Rfzupfk0bVCXhy/J4ZTDW2U6LDOzCnOC\nqYKWfrqeGyZM5+3Fn9L3iP257ZtHsu/e9TIdlpnZTnGCqUIigidz8xnx3Gwk8btzu/KtHq39pEkz\ny0pOMFXEysJN3Pz0TF6es5zeHZrzu/O60mafhpkOy8xslznBVAEvz1nOzU/PYO2GIn7SrzODv9rB\nT5o0s6znBJNBhZuK+Plzcxifu5TOBzThsUu7cdj+XjRpZtWDE0yGvLP4U66fMI2PPtvAFScdwrWn\ndqJendqZDsvMrNI4wexhm4q2MvLl9/jDa+/Tdp+GTLjsWHLae9GkmVU/TjB70LxP1nLtuNSiyQu+\n0pZbzurC3l40aWbVlL/d9oCtxcEf31jEbycvoEmDOjz0nRxO7eJFk2ZWvTnBpFn+Z6lFk2998Clf\n79KKX33rKC+aNLMawQkmTSKCp/LyGfHcHABuP+dozu3ZxosmzazGcIJJg1WFm/jxMzOZPHs5vdqn\nFk22be5Fk2ZWszjBVLK/z13OsIkzWbthCzefcTiXnnAwtb1o0sxqICeYSvL5piJ+8Zc5PPH2Ug7f\nvzFjB/ei8wFNMh2WmVnGOMFUgtzFn3L9hOks/Ww9l33tYK4/7VAvmjSzGs8JZjdsLirmrr8t4IFX\n3+fAZg0YP+RYenXwokkzM3CC2WULlq/j2nHTmPPxWs7LacNPz+pC4/p7ZTosM7MqwwlmF4ydspif\n/2UujevVYfTFPfn6EftnOiQzsyqnVjoPLqmvpPmSFkq6qYz97SS9ImmqpBmS+pWxv1DS0GS7vqS3\nJU2XNFvSiBJ1+0h6V9I0SW9I6pjGdnFip5ZMvu5EJxczs+1QRKTnwFJtYAFwGpAPvAMMjIg5JeqM\nBqZGxP2SugAvRET7EvsnAsXAWxHxW6VWKTaKiEJJewFvANdExJuSFgD9I2KupP8BekXEJTuKMScn\nJ3Jzc3e6bdv+n3nRpJnVRJLyIiKnvHrpHCLrBSyMiEVJQOOA/sCcEnUC2HYtb1Ng2bYdkgYAi4DP\n/1M59c1emGzulfy3LUNu91iVzYnFzKx86UwwrYGlJbbzgd6l6gwHXpJ0NdAIOBVAUiNgGKnez9CS\nL0h6RnlAR2BURLyV7LoUeEHSBmAtcExlNsbMzHZOOudgyvozv/R43EBgTES0AfoBYyXVAkYAIyOi\n8EsHiNgaEd2ANkAvSUcmu64D+iXH+hNwZ5lBSUMk5UrKLSgo2KWGmZlZ+dLZg8kH2pbYbsOXh60G\nA30BImKKpPpAC1I9nXMk3Q40A4olbYyIe7e9MCJWS/on0FfScqBrid7MeOCvZQUVEaOB0ZCag9m9\nJpqZ2fakswfzDtBJUgdJdYELgEml6iwB+gBI6gzUBwoi4oSIaJ9M+N8F/DIi7pXUUlKzpH4DUkNq\n84DPgKaSDk2OexowN41tMzOzcqStBxMRRZKuAiYDtYGHI2K2pFuB3IiYBNwAPCjpOlLDZ5fEji9r\nOwB4JJmHqQVMiIjnAST9AJgoqZhUwvl+utpmZmblS9tlytlgVy9TNjOrySp6mXJaF1qamVnNVaN7\nMJIKgA938eUtgJWVGE4muS1VT3VpB7gtVdXutOWgiGhZXqUanWB2h6TcinQRs4HbUvVUl3aA21JV\n7Ym2eIjMzMzSwgnGzMzSwglm143OdACVyG2peqpLO8BtqarS3hbPwZiZWVq4B2NmZmnhBFNBkhZL\nmpk80Cw3KWsu6WVJ7yU/98l0nOXZTjuGS/ooKZtW+sFvVZWkZpKekjRP0lxJx2bjOYHttiXrzouk\nw0rEO03SWknXZtt52UE7su6cAEi6LnlI4yxJTyQPb+wg6a3knIxPbulVue/rIbKKkbQYyImIlSXK\nbgc+jYhfK/XEzn0iYlimYqyI7bRjOFAYEb/NVFy7QtIjwOsR8VDy4WgI/JgsOyew3bZcSxael22S\nWzp9ROrmtVeShecFvtSO75Fl50RSa1IPZ+wSERskTQBeIHUH+6cjYpykB4DpEXF/Zb63ezC7pz/w\nSPL7I8CADMZSo0hqApwI/BEgIjZHxGqy8JzsoC3Zrg/wfkR8SBaelxJKtiNb1QEaSKpD6o+Xj4FT\ngKeS/Wk5J04wFRekHo6WJ2lIUtYqIj4GSH7ul7HoKq6sdgBcJWmGpIer+vBF4mCgAPiTpKmSHlLq\nQXXZeE621xbIvvNS0gXAE8nv2XhetinZDsiycxIRHwG/JXX3+o+BNaQe2rg6IoqSavmkHhJZqZxg\nKu74iOgBnAFcKenETAe0i8pqx/3AIUA3Uv8Af5fB+CqqDtADuD8iupN6tPZNmQ1pl22vLdl4XgBI\nhvnOBp7MdCy7o4x2ZN05SZJgf6ADcCCppwefUUbVSp8vcYKpoIhYlvxcATwD9AKWSzoAIPm5InMR\nVkxZ7YiI5cmTQouBB0m1rarLB/JLPGTuKVJf0ll3TthOW7L0vGxzBvBuRCxPtrPxvECpdmTpOTkV\n+CAiCiJiC/A0cBzQLBkyg7IfCLnbnGAqQFIjSY23/Q58HZhF6gFq302qfRd4NjMRVsz22rHtg5/4\nJqm2VWkR8QmwVNJhSVEfYA5Zdk5g+23JxvNSwkC+OKyUdecl8YV2ZOk5WQIcI6mhJPHfz8orwDlJ\nnbScE19FVgGSDib11z6khjMej4jbJO0LTADakTqJ50bEpxkKs1w7aMdYUl3+ABYDl20bL6/KJHUD\nHgLqAotIXeFTiyw6J9tspy2/JzvPS0NgKXBwRKxJyrLqswLbbUe2flZGAOcDRcBU4FJScy7jgOZJ\n2aCI2FSp7+sEY2Zm6eAhMjMzSwsnGDMzSwsnGDMzSwsnGDMzSwsnGDMzSwsnGDMzSwsnGLMqRNIY\nSeeUX9Os6nOCMTOztHCCMSuHpPbJQ8AeTB7a9JKkBpL+KSknqdMiedYOki6R9GdJz0n6QNJVkq5P\n7pT8pqTmFXzfnpJeTe58PbnEvbx+IOkdSdMlTUxuAdJUqYfJ1UrqNJS0VNJekg6R9NfkOK9LOjyp\nc27yAKrpkl5Ly/88q9GcYMwqphMwKiKOAFYD3y6n/pHAhaRuhngbsD65U/IU4DvlvZmkvYB7gHMi\noifwcHIcSD0k6isR0RWYCwxObmUyHfhaUucbwOTk5oajgauT4wwF7kvq/C9wenKcs8uLyWxn1Sm/\nipmRuhvttOT3PKB9OfVfiYh1wDpJa4DnkvKZwNEVeL/DSCWpl1P3J6Q2qdvDAxwp6RdAM2BvYHJS\nPp7U/aZeIfUMk/sk7U3qzrlPJscBqJf8/BcwJnnC4dMViMlspzjBmFVMyZsAbgUakLpx4LZRgPo7\nqF9cYruYin3uBMyOiGPL2DcGGBAR0yVdApyUlE8CfpUMwfUE/kHq2R+rI6Jb6YNExOWSegNnAtMk\ndYuIVRWIzaxCPERmtusWk/oih//e9ryyzAdaSjoWUkNmko5I9jUGPk6G0S7a9oKIKATeBu4Gnk+e\nW7IW+EDSuclxJKlr8vshEfFWRPwvsBJoW8ltsBrOCcZs1/0WuELSv4EWlXngiNhMKmn9RtJ0YBqp\noS6AnwJvAS8D80q9dDwwKPm5zUXA4OQ4s0k93RDgDkkzJc0CXiM1h2NWaXy7fjMzSwv3YMzMLC08\nyW+WAZJGAceXKr47Iv6UiXjM0sFDZGZmlhYeIjMzs7RwgjEzs7RwgjEzs7RwgjEzs7RwgjEzs7T4\nf5gsrGmsiaLoAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1040f5290>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# plot CV误差曲线\n",
    "test_means = grid_search.cv_results_[ 'mean_test_score' ]\n",
    "test_stds = grid_search.cv_results_[ 'std_test_score' ]\n",
    "train_means = grid_search.cv_results_[ 'mean_train_score' ]\n",
    "train_stds = grid_search.cv_results_[ 'std_train_score' ]\n",
    "\n",
    "n_leafs = len(num_leaves_s)\n",
    "\n",
    "x_axis = num_leaves_s\n",
    "plt.plot(x_axis, test_means)\n",
    "#plt.errorbar(x_axis, -test_means, yerr=test_stds,label = ' Test')\n",
    "#plt.errorbar(x_axis, -train_means, yerr=train_stds,label = ' Train')\n",
    "plt.xlabel( 'num_leaves' )\n",
    "plt.ylabel( 'AUC' )\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 0.84382687,  0.84432275,  0.84522116,  0.84522116])"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_means"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 性能抖动，取系统推荐值：70"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3. min_child_samples\n",
    "叶子节点的最小样本数目\n",
    "\n",
    "叶子节点数目：70，共2类，平均每类35个叶子节点\n",
    "每棵树的样本数目数目最少的类（稀有事件）的样本数目：8w * 4/5 * 1.4% = 840\n",
    "所以每个叶子节点约840/35 = 25个样本点\n",
    "\n",
    "搜索范围：10-50"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fitting 5 folds for each of 4 candidates, totalling 20 fits\n",
      "[CV] min_child_samples=10 ............................................\n",
      "[CV] min_child_samples=10 ............................................\n",
      "[CV] min_child_samples=10 ............................................\n",
      "[CV] min_child_samples=10 ............................................\n",
      "[CV] ....... min_child_samples=10, score=0.859989481507, total=   0.8s\n",
      "[CV] min_child_samples=10 ............................................\n",
      "[CV] ....... min_child_samples=10, score=0.834432287201, total=   0.8s\n",
      "[CV] min_child_samples=20 ............................................\n",
      "[CV] ....... min_child_samples=10, score=0.836587624729, total=   0.9s\n",
      "[CV] min_child_samples=20 ............................................\n",
      "[CV] ....... min_child_samples=10, score=0.848371475943, total=   1.2s\n",
      "[CV] min_child_samples=20 ............................................\n",
      "[CV] ....... min_child_samples=10, score=0.846724953086, total=   0.9s\n",
      "[CV] min_child_samples=20 ............................................\n",
      "[CV] ....... min_child_samples=20, score=0.861074829932, total=   0.9s\n",
      "[CV] min_child_samples=20 ............................................\n",
      "[CV] ....... min_child_samples=20, score=0.833236380266, total=   1.1s\n",
      "[CV] min_child_samples=30 ............................................\n",
      "[CV] ....... min_child_samples=20, score=0.832398619253, total=   1.1s\n",
      "[CV] min_child_samples=30 ............................................\n",
      "[CV] ........ min_child_samples=20, score=0.84884440359, total=   1.0s\n",
      "[CV] ....... min_child_samples=20, score=0.851392358683, total=   1.0s\n",
      "[CV] min_child_samples=30 ............................................\n",
      "[CV] min_child_samples=30 ............................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=4)]: Done  10 tasks      | elapsed:    3.7s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV] ....... min_child_samples=30, score=0.864127365232, total=   1.1s\n",
      "[CV] min_child_samples=30 ............................................\n",
      "[CV] ....... min_child_samples=30, score=0.833523237867, total=   1.0s\n",
      "[CV] min_child_samples=40 ............................................\n",
      "[CV] ....... min_child_samples=30, score=0.846271539444, total=   1.0s\n",
      "[CV] ....... min_child_samples=30, score=0.833251581582, total=   1.0s\n",
      "[CV] min_child_samples=40 ............................................\n",
      "[CV] min_child_samples=40 ............................................\n",
      "[CV] ....... min_child_samples=30, score=0.847949743861, total=   1.0s\n",
      "[CV] min_child_samples=40 ............................................\n",
      "[CV] ....... min_child_samples=40, score=0.862881495455, total=   0.9s\n",
      "[CV] min_child_samples=40 ............................................\n",
      "[CV] ....... min_child_samples=40, score=0.834211564386, total=   1.0s\n",
      "[CV] ....... min_child_samples=40, score=0.835298576573, total=   1.0s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=4)]: Done  18 out of  20 | elapsed:    6.2s remaining:    0.7s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV] ....... min_child_samples=40, score=0.848797569978, total=   1.0s\n",
      "[CV] ........ min_child_samples=40, score=0.84826059048, total=   0.9s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=4)]: Done  20 out of  20 | elapsed:    6.6s finished\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "GridSearchCV(cv=StratifiedKFold(n_splits=5, random_state=3, shuffle=True),\n",
       "       error_score='raise',\n",
       "       estimator=LGBMClassifier(boosting_type='goss',\n",
       "        categorical_feature=[0, 1, 3, 5, 6, 12, 15, 16, 17, 18, 19, 20],\n",
       "        colsample_bytree=0.7, is_unbalance=True, learning_rate=0.1,\n",
       "        max_bin=255, max_depth=6, min_child_samples=10, min_child_weight=5,\n",
       "        min_split_gain=0.0, n_estimators=38, n_jobs=4, num_leaves=70,\n",
       "        objective='binary', random_state=0, reg_alpha=0.0, reg_lambda=0.0,\n",
       "        silent=False, subsample=1.0, subsample_for_bin=50000,\n",
       "        subsample_freq=1),\n",
       "       fit_params=None, iid=True, n_jobs=4,\n",
       "       param_grid={'min_child_samples': [10, 20, 30, 40]},\n",
       "       pre_dispatch='2*n_jobs', refit=False, return_train_score='warn',\n",
       "       scoring='roc_auc', verbose=5)"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "params = {'boosting_type': 'goss',\n",
    "          'objective': 'binary',\n",
    "          'is_unbalance':True,\n",
    "          'categorical_feature': [0,1,3,5,6,12,15,16,17,18,19,20],\n",
    "          'n_jobs': 4,\n",
    "          'learning_rate': 0.1,\n",
    "          'n_estimators':n_estimators_1,\n",
    "          'num_leaves': 70,\n",
    "          'max_depth': 7,\n",
    "          'colsample_bytree': 0.7,\n",
    "         }\n",
    "lg = LGBMClassifier(silent=False,  **params)\n",
    "\n",
    "min_child_samples_s = range(10,50,10) \n",
    "tuned_parameters = dict( min_child_samples = min_child_samples_s)\n",
    "\n",
    "grid_search = GridSearchCV(lg, n_jobs=4,  param_grid=tuned_parameters, cv = kfold, scoring=\"roc_auc\", verbose=5, refit = False)\n",
    "grid_search.fit(X_train , y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.845889972267\n",
      "{'min_child_samples': 40}\n"
     ]
    }
   ],
   "source": [
    "# examine the best model\n",
    "print(grid_search.best_score_)\n",
    "print(grid_search.best_params_)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAD8CAYAAABpcuN4AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3Xd8VGXa//HPlUZICKEk1AQSIHRp\nCaAUBSmLDVDByoqrLhawIG732VX3WX+77iK6giiWxa6AouhaQMRCUxI6AkkIgYRQQk+A9Ov3xwz7\nZGMIAyQ5M5nr/Xrxysw5Z865bg7MN/c9c+4jqooxxhhzJgFOF2CMMca7WVAYY4ypkgWFMcaYKllQ\nGGOMqZIFhTHGmCpZUBhjjKmSBYUxxpgqWVAYY4ypkgWFMcaYKgU5XUB1iIqK0ri4OKfLMMYYn5KS\nknJQVaPPtl2dCIq4uDiSk5OdLsMYY3yKiOzyZDsbejLGGFMlCwpjjDFVsqAwxhhTJQsKY4wxVbKg\nMMYYUyULCmOMMVWyoDDGGFMlCwpjjPFBBcWlPLZoC9lHTtb4sSwojDHGB725ehdzV2ay58ipGj+W\nBYUxxviYE4UlzP56B4M6RNG/XdMaP54FhTHG+Ji5KzM5dKKIh0d2rJXjWVAYY4wPOV5QzJxvM7i8\nczP6tGlcK8e0oDDGGB/yync7OXaqmIdH1E5vAiwojDHGZxw9WcSry3cyqlsLureOrLXjWlAYY4yP\nmPNtBvlFJUytxd4EWFAYY4xPOJhfyL9WZHJ1j1Z0ahFRq8e2oDDGGB/wwtc7KCwp5aHhCbV+bAsK\nY4zxcvuPF/DG6l1c2zuG9tENav34FhTGGOPlZi1Lp7RMeXBY7fcmwILCGGO8WvaRk7zzw27GJ8XS\npmmYIzVYUBhjjBeb+VU6gnD/5R0cq8GCwhhjvFTmwRPMT8nmlv5taNWovmN1WFAYY4yX+ufSNIIC\nhPuGtHe0DgsKY4zxQukH8vlw/R4mDoijWcNQR2uxoDDGGC/0zJephAYHcvel7ZwuxbOgEJFRIrJd\nRNJF5LeVrG8jIstEZJ2IbBSRKytZny8ij5Rblikim0RkvYgkl1veS0RWn14uIv0upIHGGONrtu49\nzicb9/KLgXE0bVDP6XIIOtsGIhIIzAJGANnAGhFZpKo/ltvsUWCeqs4Wka7Ap0BcufUzgM8q2f1Q\nVT1YYdlTwOOq+pk7cJ4ChnjYHmOM8XkzlqQSERrEpMHOfjZxmic9in5AuqpmqGoR8C4wpsI2CjR0\nP44Eck6vEJGxQAawxcOazrgvY4yp6zZlH2Pxj/u5a1A7IsOCnS4H8KBHAbQGsso9zwb6V9jmMWCx\niNwPhAPDAUQkHPgNrt7IIxVeo+7XKPCiqs5xL38I+EJE/oEryAZ43BpjjPFx05dsp1FYMHcMinO6\nlP/wpEchlSzTCs9vBuaqagxwJfCGiAQAjwMzVDW/kn0MVNU+wBXAZBG51L38XmCqqsYCU4FXKi1K\nZJL7M4zk3NxcD5phjDHeLWXXYb7ensvdl7YnItQ7ehPgWVBkA7Hlnsfw0+GgO4F5AKq6CggFonD1\nPJ4SkUxcPYXfi8gU93Y57p8HgIW4hrgAJgIfuB/PL7f8v6jqHFVNUtWk6OhoD5phjDHebfriVKIa\nhDBxQFunS/kvngTFGiBBROJFJAS4CVhUYZvdwDAAEemCKyhyVXWwqsapahzwDPCkqs4UkXARiXBv\nHw6MBDa795UDXOZ+fDmQdt6tM8YYH7FqxyFW7jjEvUM6EBbiyacCtees1ahqibsX8AUQCLyqqltE\n5AkgWVUXAdOAl0RkKq5hqdtVteLwVHnNgYUicrqGt1X1c/e6XwLPikgQUABMOs+2GWOMT1BVnl6y\nneYN63Fr/zZOl/MTHsWWqn6K6yuv5Zf9sdzjH4GBZ9nHY+UeZwA9z7DdciDRk7qMMaYu+DbtIGsy\nj/DnMd0IDQ50upyfsCuzjTHGQarK04u307pRfW7oG3v2FzjAgsIYYxz05dYDbMg+xgPDOlAvyPt6\nE2BBYYwxjikrU55ekkpc0zCu6xPjdDlnZEFhjDEO+XzLPrbuPc6DwxMIDvTet2PvrcwYY+qwUndv\nokOzBozu2drpcqpkQWGMMQ74eEMO6QfymTq8I4EBlU2A4T0sKIwxppaVlJbxzJepdG4RwRXdWzhd\nzllZUBhjTC37YO0eMg+dZNrITgR4eW8CLCiMMaZWFZWU8ezSNHrGRDK8SzOny/GIBYUxxtSi95Kz\n2HP0FFNHdMQ9jZHXs6AwxphaUlBcysyv0khq25jLOvrOrNcWFMYYU0ve+n43+48X8vBI3+lNgAWF\nMcbUipNFJcz+Op0B7ZsyoH2U0+WcEwsKY4ypBa+v2sXB/CKmjezodCnnzILCGGNqWF5BMS98s4PL\nOkaT2LaJ0+WcMwsKY4ypYf9akcnRk8U+2ZsACwpjjKlRx04W89J3GYzo2pweMY2cLue8WFAYY0wN\neum7DPIKSnh4hG/2JsCCwhhjasyh/EL+tWInV/VoSZeWDZ0u57xZUBhjTA158dsMThWXMnV4gtOl\nXBALCmOMqQEH8gp4fVUmY3u1pkOzCKfLuSAWFMYYUwOeX7aD4lLlgWG+3ZsACwpjjKl2OUdP8fb3\nuxnXJ4a4qHCny7lgFhTGGFPNZi5LR1HuH9bB6VKqhQWFMcZUo92HTjJvTRY39W1DTOMwp8upFhYU\nxhhTjf75VRqBAcKUy+tGbwIsKIwxptpk5ObzwdpsJlzcluYNQ50up9pYUBhjTDV55ss06gUFcu+Q\n9k6XUq0sKIwxphps35fHxxtzuH1gHFEN6jldTrWyoDDGmGowY0kq4SFBTBrczulSqp0FhTHGXKDN\ne47x+ZZ93DkonsbhIU6XU+0sKIwx5gLNWJJKZP1g7hwc73QpNcKCwhhjLsDa3UdYuu0Aky5tR8PQ\nYKfLqREWFMYYcwFmLEmlSXgItw+Ic7qUGmNBYYwx5+n7jEN8l3aQey9rT3i9IKfLqTEWFMYYcx5U\nlelLUomOqMeEi9s6XU6NsqAwxpjzsCL9ED/sPMyUoR2oHxLodDk1yqOgEJFRIrJdRNJF5LeVrG8j\nIstEZJ2IbBSRKytZny8ij5Rblikim0RkvYgkV9j+fvfxtojIU+fbOGOMqQmqyj8Wb6dVZCg39Yt1\nupwad9ZBNREJBGYBI4BsYI2ILFLVH8tt9igwT1Vni0hX4FMgrtz6GcBnlex+qKoerHC8ocAYoIeq\nFopIs3NpkDHG1LRl2w+wPuso/++6i6gXVLd7E+BZj6IfkK6qGapaBLyL6428PAVO3zk8Esg5vUJE\nxgIZwBYPa7oX+KuqFgKo6gEPX2eMMTVOVZm+OJU2TcIYlxjjdDm1wpOgaA1klXue7V5W3mPABBHJ\nxtWbuB9ARMKB3wCPV7JfBRaLSIqITCq3vCMwWES+F5FvRKRvZUWJyCQRSRaR5NzcXA+aYYwxF+6L\nLfvYknOcB4YlEBzoHx/zetJKqWSZVnh+MzBXVWOAK4E3RCQAV0DMUNX8SvYxUFX7AFcAk0XkUvfy\nIKAxcDHwK2CeiPykBlWdo6pJqpoUHR3tQTOMMebClJYpTy9JpV10OGN7tXK6nFrjyRd/s4Hyn9bE\nUG5oye1OYBSAqq4SkVAgCugPjHN/IN0IKBORAlWdqao57u0PiMhCXENc37qP94GqKvCDiJS592Xd\nBmOMoz7ZmEPq/nz+eXNvgvykNwGe9SjWAAkiEi8iIcBNwKIK2+wGhgGISBcgFMhV1cGqGqeqccAz\nwJOqOlNEwkUkwr19ODAS2Oze14fA5e51HYEQ4L8+8DbGmNpWUlrGs1+m0al5BFdf1NLpcmrVWXsU\nqloiIlOAL4BA4FVV3SIiTwDJqroImAa8JCJTcQ1L3e7uEZxJc2Che0QpCHhbVT93r3sVeFVENgNF\nwMSz7MsYY2rch+tzyDh4ghcmJBIQUNmIfN0ldeE9OCkpSZOTk8++oTHGnIfi0jIun/41kfWD+XjK\nICr52NQniUiKqiadbTv/GWQzxpjzND85m6zDp5g2olOdCYlzYUFhjDFVKCgu5bmv0ujdphFDOvnn\nNywtKIwxpgrv/rCbvccKeGSkf/YmwILCGGPO6FRRKbO+3kH/+CYMaN/U6XIcY0FhjDFn8MbqTHLz\nCpnmx70JsKAwxphK5ReW8MI3GQxOiKJffBOny3GUBYUxxlTitZWZHD5RxLSRnZwuxXEWFMYYU8Gx\nU8W8+M0OhndpRq/YRk6X4zgLCmOMqeCV5Ts5XlDC1BEdnS7FK1hQGGNMOUdOFPHq8p1c0b0F3VpF\nOl2OV7CgMMaYcl78NoMTRdabKM+Cwhhj3HLzCnltZSaje7aiY/MIp8vxGhYUxhjjNvvrHRSWlPLg\nsASnS/EqFhTGGAPsO1bAm9/v4vo+MbSLbuB0OV7FgsIYY4CZy9IoK1MesN7ET1hQGGP8XvaRk7y3\nJosb+8YS2yTM6XK8jgWFMcbvPbc0HRFhyuUdnC7FK1lQGGP8WubBEyxYm80t/drQMrK+0+V4JQsK\nY4xfe3ZpGsGBwn1D2ztditeyoDDG+K20/Xl8uH4PEy+Jo1lEqNPleC0LCmOM33rmyzTCggO5+zLr\nTVTFgsIY45d+zDnOvzft5Y5B8TQJD3G6HK9mQWGM8UtPL0klIjSIuwa1c7oUr2dBYYzxOxuyjvLl\n1v1MGtyOyLBgp8vxehYUxhi/M31JKo3DgvnFoHinS/EJFhTGGL+SnHmYb1Nzueey9jSoF+R0OT7B\ngsIY41emL04lqkE9brskzulSfIYFhTHGb6xMP8iqjEPcN6Q99UMCnS7HZ1hQGGP8gqoyfUkqLRqG\nckv/Nk6X41MsKIwxfuGb1FxSdh1hyuUdCA223sS5sKAwxtR5qsrTS1KJaVyfG5JinS7H51hQGGPq\nvCU/7mdj9jEeGJZASJC97Z0r+xszxtRpZWWu3kR8VDjX9W7tdDk+yYLCGFOnfbp5L9v25fHQ8ASC\nAu0t73zY35oxps4qLVNmLEkloVkDru7RyulyfJYFhTGmzvpo/R525J5g6oiOBAaI0+X4LAsKY0yd\nVFxaxrNL0+jasiGjurVwuhyf5lFQiMgoEdkuIuki8ttK1rcRkWUisk5ENorIlZWszxeRR8otyxSR\nTSKyXkSSK9nnIyKiIhJ1Pg0zxvi391Oy2XXoJA+P6EiA9SYuyFlnxBKRQGAWMALIBtaIyCJV/bHc\nZo8C81R1toh0BT4F4sqtnwF8Vsnuh6rqwUqOGes+3m5PG2KMMacVlpTy3Ffp9IxtxLAuzZwux+d5\n0qPoB6SraoaqFgHvAmMqbKNAQ/fjSCDn9AoRGQtkAFvOoa4ZwK/d+zXGmHMyb00We46eYtqIjohY\nb+JCeRIUrYGscs+z3cvKewyYICLZuHoT9wOISDjwG+DxSvarwGIRSRGRSacXishoYI+qbqiqKBGZ\nJCLJIpKcm5vrQTOMMf6goNjVm+gb15jBCTZyXR08CYrK4rjib/o3A3NVNQa4EnhDRAJwBcQMVc2v\nZB8DVbUPcAUwWUQuFZEw4A/AH89WlKrOUdUkVU2Kjo72oBnGVxSXljldgvFhb67exYG8QqaN7GS9\niWriyV07soHyk6PEUG5oye1OYBSAqq4SkVAgCugPjBORp4BGQJmIFKjqTFXNcW9/QEQW4hriOgLE\nAxvcJzgGWCsi/VR13/k20ni/U0WlfLZ5L/OTs1m98xA3JMbyxNhu1AuyyduM504UljD76x0M7NCU\ni9s1dbqcOsOToFgDJIhIPLAHuAm4pcI2u4FhwFwR6QKEArmqOvj0BiLyGJCvqjPdQ1IBqprnfjwS\neEJVNwHNyr0mE0iq7ANv4/tUlZRdR1iQks0nG/eSX1hC26ZhXNOjFe8lZ7EjN5/ZExKJjqjndKnG\nR7y2KpNDJ4p4eEQnp0upU84aFKpaIiJTgC+AQOBVVd0iIk8Ayaq6CJgGvCQiU3ENS92uqlV9EN0c\nWOjuNQQBb6vq5xfYFuMj9h0r4P212byfkk3GwROEhQRy5UUtGZ8YQ7/4JogII7s155H5Gxgzczlz\nbkuie+tIp8s2Xu54QTEvfpPB0E7RJLZt7HQ5dYpU/X7uG5KSkjQ5+SeXYhgvUlBcypdb9zM/OZvv\n0nIpU+gX34TxiTFceVFLwiu5d/HmPceY9Hoyh08WMX18L67q0dKByo2veObLVJ75Mo2Ppwziohj7\nxcITIpKiqkln287uLG5qjKqyec9x5qdk8dH6HI6dKqZVZCiTh3ZgXGIMbZuGV/n67q0j+WjKIO55\nM4XJb69l+74OPDTcLp4yP3X0ZBGvfLeTn3VrbiFRAywoTLU7mF/Ih+v2sCAlm2378qgXFMDPurVg\nfFIMA9pHndOcO9ER9Xj7l/35nw8388+v0tm2L48ZN/aqtAdi/NdL32WQX1TC1BEdnS6lTrL/baZa\nFJeWsWzbAeanZLNs2wFKypSesY3437HduaZnKyLrB5/3vusFBfK363vQuUVD/vffP3L97JW8dFsS\nsU3CqrEFxlcdyi/kXysyubpHKzq3aHj2F5hzZkFhLsj2fXnMT87iw/V7OJhfRHREPe4cFM+4xBgS\nmkdU23FEhDsGxdOhWQOmvL2WMbNW8PytfewrkIYXvtlBQXEpDw1PcLqUOsuCwpyzYyeLWbRhD/NT\nstmYfYzgQGFY5+aMT4rhso7RNXpzmEs7RvPRlEHc9doaJrz8PY+P6cat/dvW2PGMd9t/vIDXV+1i\nbO/WtI9u4HQ5dZYFhfFIaZnyXVouC1KyWfzjfopKyujSsiF/vLorY3u3pkl4SK3VEh8VzsLJA3nw\nnXX8YeFmtu3N44/XdCXY7l7md55flk5pmfLgMOtN1CQLClOljNx8FqRk88HaPew7XkDjsGBu6deG\ncYkxjl7b0DA0mJcn9uWpL7bx4jcZpB3I4/lbE2s1sIyz9hw9xTs/ZDE+6ezfoDMXxoLC/ER+YQn/\n3pjD/ORskncdIUBgSKdm/OmarlzepZnXTKsRGCD87ooudG4RwW/e38SYWct5+ba+dGpRfZ+NGO81\n86s0AKZcbr2JmmZBYQAoK1O+33mY+SlZfLZpH6eKS2kfHc5vr+jMdb1b06xhqNMlntG1vWOIj2rA\npNeTue75Fcy4sRcj7Y5mddquQyeYn5zNrf3b0LpRfafLqfMsKPxc1uGTruk01maTdfgUEfWCGNu7\nNeOTYugd28hnZt/sFduIRVMGMemNZCa9kcIjIzsyeWgHn6nfnJtnl6YRGCBMHtrB6VL8ggWFHzpV\nVMrnW1wzta7ccQgRGNg+imkjOvGzbi2oH+IdQ0vnqkVkKPPuvoTfvL+RfyxOZdu+PP4+rqfPtsdU\nLv1APh+u28Odg+K9uqdbl1hQ+AlVZe3uoyxIyeKTDXvJKyyhTZMwHh7Rkev6tCamcd24eC00OJBn\nbuxF5xYNeeqLbWQeOsGcnyfRyoYn6oxnl6YRGhzIPZe1d7oUv2FBUcftP+6aqXVBSjYZuSeoH+ye\nqTUphn5xTerkvEkiwr1D2tOxeQMefHc9o2eu4MWfJ9qMonXAtn3H+XhDDvcNaU/TBjb9fG2xoKiD\nCktK+fLHA8xPyeLbVNdMrX3jGnPPpe25skdLGvjJPEnDujRn4X0DuOv1ZG6es5q/XNud8UmxZ3+h\n8VozlqQSUS+ISZe2c7oUv+If7xh+QFXZknOc+clZfLQhh6Mni2kZGcq9Q9ozLjGW+Cj//J55QvMI\nPpo8kMlvr+VXCzaybV8ev7uic41ePW5qxqbsY3yxZT8PDU+gUZhdL1ObLCh83KH8Qj5cn8P85Cy2\n7csj5PRMrYkxDOxwbjO11lWNwkJ47Rf9+N9/b+WV5TtJ3Z/HzJv7EBl2/hMVmtr39JLtNAoL5o5B\n8U6X4ncsKHxQcWkZ32zPZX5KFku3umdqjYnkz2O7M7pHK3sDrERQYACPje5G5xYR/M9Hm7n2+RW8\nNDHJ5gfyESm7jrBsey6/HtWJhqH277u2WVD4kNT9rplaF67L4WB+IVENQvjFwDjGJ8XSsRpnaq3L\nburXhvbNGnDPGymMnbWCf97cm6Gdmp39hcZRTy/ZTtPwECZeEud0KX7JgsLLHTtZzKKNOSxIzmJD\n9jGCAoTLOzdjfFIsQzpF20R456FvXBM+mjKQX76ewp1z1/C7K7pw1+B4uzjPS63OOMSK9EM8elUX\nu2GVQ+xv3QuVlikr0g8yPyWbL7bso6ikzDVkcnVXxvRqRZR9LfCCxTQO4/17L2HavA385dOtbN13\nnCevvYjQYLs4z5uoKk8vTqV5w3pMuNimk3eKBYUXyTx4ggUpruk09h4roFFYMDf3jWV8UizdWjW0\n33irWVhIELNu6cNzX6Uz48tUdh48wYsTEu1qXy/yXdpBfsg8zJ/HdLMQd5AFhcPyC0v4dONeFqRk\n80PmYQLEdXOeR6/qyvCu3jNTa10VECA8ODyBTi0aMPW9DYyeuYI5tyXSI6aR06X5PVVl+pJUWjeq\nzw197foXJ1lQOEDVPVNrcjafbd7LyaJS2kWF8+tRnbiudwwtIu032to2qntL2jQJ55evJzP+hVU8\nNa4HY3q1drosv7Z06wE2ZB3lr9ddZL8wOcyCohbtOXqK91Nc02nsPnySBvWCGNOrFeMSY+nTxndm\naq2rurZqyKIpA7n3rbU8+O56tu/L45GRnerkNCferqxMeXpJKm2bhnF9YozT5fg9C4oaVlBcyhdb\n9jE/OZsVOw6iCgPaN2XqiARGdWtpM5t6maYN6vHmnf3506ItPP/1DlL35zHjxl5E2Hf3a9UXW/bx\n497jPH1DT/tmnxewoKgBqsq6rKPMT87mkw055BWWENO4Pg8OS+D6PjHENqkbM7XWVSFBATx5bXe6\ntIzg8Y9/5LrnV/LyxCS73WYtKXX3JtpHh9vwn5ewoKhGB44X8MG6PSxIySb9QD71gwO54qIWjEuM\n4eL4pjaE4UNEhNsuiaNDdAPue3sto2eu4Plb+zCwQ5TTpdV5n2zMIe1APjNv6W1T0HgJUVWna7hg\nSUlJmpyc7Mixi0rKWLp1P/NTsvkmNZfSMiWpbWPGJ8Vw5UUtbciiDth16AR3vZZMxsET/PHqrtx2\nSVv7PKmGlJSWMWLGt9QLCuDTBwbbL1c1TERSVDXpbNtZj+I8bd5zjAUp2Xy0fg9HThbTomEod1/a\njnGJMbSz+YPqlLZNw/ngvgFMfW89f1q0hW37jvP46O6EBNnYeXX7YN0edh48wZyfJ1pIeBELinNw\n+EQRH67bw/yUbLbuPU5IYAAjujVnfGIMgxOirZtch0WEBjPn50lMX7KdWct2sOPACWZP6GM3z6lG\nRSVl/HNpGj1iIhnRtbnT5ZhyLCjOoqS0jG9Sc5mfnM3SbfspLlV6xETyxJhujO7ZyubF9yMBAcKv\nftaZjs0j+PWCjYyeuYKXbkuia6uGTpdWJ8xLziL7yCn+PLa7De15GQuKM0g/kMf85Gw+WLeH3LzC\n/8xcOS4phs4t7I3Bn43p1Zr4qHAmvZ7C9bNX8vQNPbniopZOl+XTCopLmflVOoltGzOkY7TT5ZgK\nLCjKOXaqmE825jA/OZv1WUcJChCGdm7G+MQYhnZuZt/nNv/RI6YRi6YM5O43U7j3rbU8NDyBBy5P\nsHH18/T297vZd7yAp2/oab0JL+T3QVFWpqzYcZD5ya6ZWgtLyujUPIJHr+rC2N6tbaZWc0bNGoby\nzi8v5vcLN/HMl2ls35fH9Bt6Ehbi9/+tzsmpolKe/3oHl7RrygD7+rFX8ut/0W+s3sXsZenkHCsg\nsn4wN/aNZXxiLN1b20ytxjOhwYFMH9+Tri0b8uSnW8mcfZKXbkskprFdVOmp11dlcjC/kBcm9HG6\nFHMGfh0Up4pKSGgewe+v6sLwLs1tGmNzXkSEuwa3o0OzBtz/zjrGzFzB7AmJ9Itv4nRpXi+/sIQX\nvtnBpR2jSYqzvy9v5dGgu4iMEpHtIpIuIr+tZH0bEVkmIutEZKOIXFnJ+nwReaTcskwR2SQi60Uk\nudzyv4vINvd+FopIjc33/MvB7Xjtjn5c3aOVhYS5YEM6NePDyQOJrB/MrS+v5p0fdjtdktf71/Kd\nHDlZzLQRHZ0uxVThrEEhIoHALOAKoCtws4h0rbDZo8A8Ve0N3AQ8X2H9DOCzSnY/VFV7VbgycAnQ\nXVV7AKnA7zxqyXmw4SVT3dpHN2DhfQO5pH0Uv/tgE3/6aDMlpWVOl+WVjp0sZs53GQzv0pyesXb/\nD2/mSY+iH5CuqhmqWgS8C4ypsI0Cp78zGgnknF4hImOBDGCLJwWp6mJVLXE/XQ3YHMPGp0SGBfPq\nxCTuGhTPa6t2MfFfP3D0ZJHTZXmdl5dnkFdQwsPWm/B6ngRFayCr3PNs97LyHgMmiEg28ClwP4CI\nhAO/AR6vZL8KLBaRFBGZdIZj30HlPRFjvFpQYACPXt2Vv4/rwZqdRxgzawVp+/OcLstrHD5RxKvL\nd3LVRS3tgkUf4ElQVDY+U3EmwZuBuaoaA1wJvCEiAbgCYoaq5leyj4Gq2gfXkNZkEbn0vw4q8geg\nBHir0qJEJolIsogk5+bmetAMY2rf+KRY3pl0MScKS7n2+ZUs3brf6ZK8wovf7OBkcSkPDU9wuhTj\nAU+CIhsof8PaGMoNLbndCcwDUNVVQCgQBfQHnhKRTOAh4PciMsW9XY775wFgIa4hLgBEZCJwNXCr\nnmF6W1Wdo6pJqpoUHW1Xchrvldi2MR/fP5C4qDDuej2Z2V/voC7M2ny+DuQV8NqqTMb2ak1C8win\nyzEe8CQo1gAJIhIvIiG4PqxeVGGb3cAwABHpgisoclV1sKrGqWoc8AzwpKrOFJFwEYlwbx8OjAQ2\nu5+PwjVcNVpVT15wC43xAi0j6zP/7gFcdVFL/vb5Nh56bz0FxaVOl+WI2V/voLhUeXCY9SZ8xVmv\no1DVEncv4AsgEHhVVbeIyBNAsqouAqYBL4nIVFzDUrefqSfg1hxY6P7WURDwtqp+7l43E6gHLHGv\nX62q95xf84zxHvVDAnnu5t50admQv3+x3T2ddhItIkOdLq3W7D12irdW72ZcnxjiouyOgb7Cblxk\njAMWb9nH1PfWE1YviDk/T6S3Hd1dAAAL+ElEQVR3m8ZOl1Qr/rBwE/OSs1j2yBC7et0LeHrjIpvl\nzhgHjOzWgg/uG0hocAA3zlnNB2uznS6pxmUdPsl7a7K4sW+shYSPsaAwxiGdWkSwaPIgEts05uF5\nG/h/n26ltMz3e/hn8s+laQQECFOG2mcTvsaCwhgHNQ4P4fU7+3HbJW158dsM7nxtDccLip0uq9pl\n5Obzwbo9TOjf1q8+k6krLCiMcVhwYABPjOnOX67tzvK0g4ydtYKM3MouPfJdzy5NIyQwgHuHtHe6\nFHMeLCiM8RK39m/Lm3f158iJIsbOWsG3qXXjQtLU/Xks2pDDxAFxREfY/V18kQWFMV7k4nZNWTRl\nEK0a1ef2f/3AK8t3+vzFeTOWpBIeEsTdl7ZzuhRzniwojPEysU3CeP/eAYzo2pw/f/Ijv16wkcIS\n37w4b0vOMT7bvI87BsXTODzE6XLMebKgMMYLhdcLYvatiTwwLIH5KdncPGc1B/IKnC7rnM1YkkrD\n0CDuHBTvdCnmAlhQGOOlAgKEh0d0ZNYtffhx73HGzFzB5j3HnC7LY+t2H+HLrQeYdGk7IusHO12O\nuQAWFMZ4uat6tGTBPQMQYNwLK/l4Q8U5Ob3T00tSaRIewu0DrTfh6ywojPEB3VtH8tGUQXRvFcn9\n76xj+uLtlHnxxXk/7DzMd2kHueeydjSod9Yp5YyXs6AwxkdER9TjrV/258akWJ77Kp2730whv7Dk\n7C+sZarK9MXbiY6ox88vjnO6HFMNLCiM8SH1ggL56/UX8adrurJ0636uf34lWYe9azb+lTsO8f3O\nw0we0p76IYFOl2OqgQWFMT5GRPjFwHheu6Mfe4+dYvTM5azaccjpsgBXb+Ifi7fTMjKUm/q1cboc\nU00sKIzxUYMTovloyiCahIfw81e+543Vu5wuia+357Ju91HuvzyB0GDrTdQVFhTG+LD4qHAWTh7I\n4IQo/ufDzfxh4SaKS8scqUVVmb5kO7FN6jM+KcaRGkzNsKAwxsc1DA3m5Yl9ufuydrz1/W4mvPw9\nh08U1XodX2zZz+Y9x3lwWEeCA+2tpS6xs2lMHRAYIPzuii7MuLEn67KOMnrmcrbuPV5rxy8rU2Ys\nSaVdVDhje7WqteOa2mFBYUwdcm3vGObdfQlFJWVcP3slX2zZVyvH/WTTXrbvz+PB4QkEWW+izrEz\nakwd0yu2ER/fP4iEZg24+40UnluaVqMz0JaUlvHMl6l0ah7BNT2sN1EXWVAYUwc1bxjKe3dfwrW9\nWzN9SSpT3lnHqaKamYH2o/U5ZOSeYOqIBAICpEaOYZxl19YbU0eFBgfy9A096dQigr99vo3Mgyd4\n6bYkWjWqX23HKC4t49mlaXRr1ZCfdWtRbfs13sV6FMbUYSLCPZe155WJSew6dJLRM5eTsutwte1/\nQUo2uw+fZNrIjohYb6KusqAwxg9c3rk5H04eQIN6Qdw0ZzXz1mRd8D4LS0p5bmkavWIbMbRTs2qo\n0ngrCwpj/ESHZhF8OHkg/eOb8uv3N/LExz9ScgEX5737QxY5xwp4ZGQn603UcRYUxviRRmEhzP1F\nX34xMI5XV+zkF3PXcOxk8Tnv51RRKTOXpdMvvgkDOzStgUqNN7GgMMbPBAUG8KdruvG36y9idcYh\nxsxaTvqB/HPax5urd5GbV8i0EfbZhD+woDDGT93Ytw3v/PJi8gtLuHbWCpZtP+DR604UljD7mx0M\nToiifzvrTfgDCwpj/FhSXBM+mjKI2CZh3DF3DXO+3XHWi/Pmrszk8IkiHh7RsZaqNE6zoDDGz7Vu\nVJ8F917Cld1b8uSn25g2bwMFxZVfnHe8oJg532YwrHMzerdpXMuVGqdYUBhjCAsJYuYtvXl4REc+\nWLeHG+esZv/xgp9s98p3Ozl2qpip1pvwKxYUxhjAdXHeA8MSeGFCImn78xg9czkbso7+Z/2RE0W8\nsnwno7q1oHvrSAcrNbXNgsIY819GdW/B+/cOICgggPEvruLDdXsAmPNdBieKSqw34YdsridjzE90\nadmQRVMGcu9ba3novfWszzrKe2uyuKZHKzq1iHC6PFPLrEdhjKlU0wb1ePPO/tzSvw1zV2ZSWFLK\nQ8MTnC7LOMB6FMaYMwoJCuDJay8isU1jCkvKaBfdwOmSjAMsKIwxZ3V9YozTJRgHeTT0JCKjRGS7\niKSLyG8rWd9GRJaJyDoR2SgiV1ayPl9EHim3LFNENonIehFJLre8iYgsEZE090/7srYxxjjorEEh\nIoHALOAKoCtws4h0rbDZo8A8Ve0N3AQ8X2H9DOCzSnY/VFV7qWpSuWW/BZaqagKw1P3cGGOMQzzp\nUfQD0lU1Q1WLgHeBMRW2UaCh+3EkkHN6hYiMBTKALR7WNAZ4zf34NWCsh68zxhhTAzwJitZA+buc\nZLuXlfcYMEFEsoFPgfsBRCQc+A3weCX7VWCxiKSIyKRyy5ur6l4A90+7I4oxxjjIk6CobA7hirOG\n3QzMVdUY4ErgDREJwBUQM1S1sjmMB6pqH1xDWpNF5NJzqBsRmSQiySKSnJubey4vNcYYcw48CYps\nILbc8xjKDS253QnMA1DVVUAoEAX0B54SkUzgIeD3IjLFvV2O++cBYCGuIS6A/SLSEsD9s9K5j1V1\njqomqWpSdHS0B80wxhhzPjwJijVAgojEi0gIrg+rF1XYZjcwDEBEuuAKilxVHayqcaoaBzwDPKmq\nM0UkXEQi3NuHAyOBze59LQImuh9PBD4679YZY4y5YGe9jkJVS9y9gC+AQOBVVd0iIk8Ayaq6CJgG\nvCQiU3ENS92uVU9q3xxY6L4zVhDwtqp+7l73V2CeiNyJK4DGn2fbjDHGVAM5201KfIGI5AK7zvPl\nUcDBaizHSdYW71NX2gHWFm91IW1pq6pnHbuvE0FxIUQkucJ1HD7L2uJ96ko7wNrirWqjLTYpoDHG\nmCpZUBhjjKmSBQXMcbqAamRt8T51pR1gbfFWNd4Wv/+MwhhjTNWsR2GMMaZKfhUUIvKqiBwQkc3l\nlvnktOZnaMtjIrLHPXX7+orTvXsjEYl1T1G/VUS2iMiD7uU+d16qaIsvnpdQEflBRDa42/K4e3m8\niHzvPi/vuS/C9VpVtGOuiOwsd056OV2rp0Qk0H1Lh0/cz2v8nPhVUABzgVEVlvnqtOZz+WlbwDW3\nVi/3n09ruabzUQJMU9UuwMW45v3qim+elzO1BXzvvBQCl6tqT6AXMEpELgb+hqstCcARXNP3eLMz\ntQPgV+XOyXrnSjxnDwJbyz2v8XPiV0Ghqt8Chyss9slpzc/QFp+jqntVda37cR6u/wCt8cHzUkVb\nfI66nJ7MM9j9R4HLgQXu5V5/Xqpoh08SkRjgKuBl93OhFs6JXwXFGdS1ac2nuO8y+KovDNeUJyJx\nQG/ge3z8vFRoC/jgeXEPcazHNTHnEmAHcFRVS9ybVHbLAa9TsR2qevqc/MV9TmaISD0HSzwXzwC/\nBsrcz5tSC+fEgqJumQ20x9XF3gtMd7Ycz4lIA+B94CFVPe50PReikrb45HlR1VJV7YVrxuh+QJfK\nNqvdqs5dxXaISHfgd0BnoC/QBNd9c7yaiFwNHFDVlPKLK9m02s+JBYWH05r7AlXd7/5PUQa8xP9N\n3e7VRCQY1xvrW6r6gXuxT56Xytriq+flNFU9CnyN63OXRiJyejLRym454LXKtWOUe5hQVbUQ+Be+\ncU4GAqPdt214F9eQ0zPUwjmxoKhD05qffmN1u5b/m7rda7nHWF8Btqrq0+VW+dx5OVNbfPS8RItI\nI/fj+sBwXJ+5LAPGuTfz+vNyhnZsK/dLiOAa0/f6c6Kqv1PVGPdtG24CvlLVW6mFc+JXF9yJyDvA\nEFyzLe4H/gR8iOumS21wT2uuql7/IfEZ2jIE1/CGApnA3afH+b2ViAwCvgM28X/jrr/HNbbvU+el\nirbcjO+dlx64PhgNxPUL5TxVfUJE2uH6bbYJsA6Y4P6t3CtV0Y6vgGhcQzfrgXvOcCdOryQiQ4BH\nVPXq2jgnfhUUxhhjzp0NPRljjKmSBYUxxpgqWVAYY4ypkgWFMcaYKllQGGOMqZIFhTHGmCpZUBhj\njKmSBYUxxpgq/X8FLMEjXN1oiwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10410d650>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# plot CV误差曲线\n",
    "test_means = grid_search.cv_results_[ 'mean_test_score' ]\n",
    "test_stds = grid_search.cv_results_[ 'std_test_score' ]\n",
    "train_means = grid_search.cv_results_[ 'mean_train_score' ]\n",
    "train_stds = grid_search.cv_results_[ 'std_train_score' ]\n",
    "\n",
    "x_axis = min_child_samples_s\n",
    "\n",
    "plt.plot(x_axis, test_means)\n",
    "#plt.errorbar(x_axis, -test_scores, yerr=test_stds ,label = ' Test')\n",
    "#plt.errorbar(x_axis, -train_scores, yerr=train_stds,label =  +' Train')\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 0.84522116,  0.84538925,  0.84502473,  0.84588997])"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_means"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 再次细调"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fitting 5 folds for each of 2 candidates, totalling 10 fits\n",
      "[CV] min_child_samples=40 ............................................\n",
      "[CV] min_child_samples=40 ............................................\n",
      "[CV] min_child_samples=40 ............................................\n",
      "[CV] min_child_samples=40 ............................................\n",
      "[CV] ....... min_child_samples=40, score=0.862881495455, total=   0.8s\n",
      "[CV] min_child_samples=40 ............................................\n",
      "[CV] ....... min_child_samples=40, score=0.834211564386, total=   0.8s\n",
      "[CV] ....... min_child_samples=40, score=0.835298576573, total=   1.0s\n",
      "[CV] min_child_samples=50 ............................................\n",
      "[CV] ....... min_child_samples=40, score=0.848797569978, total=   0.9s\n",
      "[CV] min_child_samples=50 ............................................\n",
      "[CV] min_child_samples=50 ............................................\n",
      "[CV] ........ min_child_samples=40, score=0.84826059048, total=   0.8s\n",
      "[CV] min_child_samples=50 ............................................\n",
      "[CV] ....... min_child_samples=50, score=0.863051849311, total=   0.9s\n",
      "[CV] ....... min_child_samples=50, score=0.834790030298, total=   0.9s\n",
      "[CV] min_child_samples=50 ............................................\n",
      "[CV] ......... min_child_samples=50, score=0.8328613456, total=   0.9s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=4)]: Done   6 out of  10 | elapsed:    2.7s remaining:    1.8s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV] ....... min_child_samples=50, score=0.845927519017, total=   0.5s\n",
      "[CV] ........ min_child_samples=50, score=0.84658651844, total=   0.5s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=4)]: Done  10 out of  10 | elapsed:    3.4s finished\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "GridSearchCV(cv=StratifiedKFold(n_splits=5, random_state=3, shuffle=True),\n",
       "       error_score='raise',\n",
       "       estimator=LGBMClassifier(boosting_type='goss',\n",
       "        categorical_feature=[0, 1, 3, 5, 6, 12, 15, 16, 17, 18, 19, 20],\n",
       "        colsample_bytree=0.7, is_unbalance=True, learning_rate=0.1,\n",
       "        max_bin=255, max_depth=6, min_child_samples=10, min_child_weight=5,\n",
       "        min_split_gain=0.0, n_estimators=38, n_jobs=4, num_leaves=70,\n",
       "        objective='binary', random_state=0, reg_alpha=0.0, reg_lambda=0.0,\n",
       "        silent=False, subsample=1.0, subsample_for_bin=50000,\n",
       "        subsample_freq=1),\n",
       "       fit_params=None, iid=True, n_jobs=4,\n",
       "       param_grid={'min_child_samples': [40, 50]}, pre_dispatch='2*n_jobs',\n",
       "       refit=False, return_train_score='warn', scoring='roc_auc',\n",
       "       verbose=5)"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "params = {'boosting_type': 'goss',\n",
    "          'objective': 'binary',\n",
    "          'is_unbalance':True,\n",
    "          'categorical_feature': [0,1,3,5,6,12,15,16,17,18,19,20],\n",
    "          'n_jobs': 4,\n",
    "          'learning_rate': 0.1,\n",
    "          'n_estimators':n_estimators_1,\n",
    "          'num_leaves': 70,\n",
    "          'max_depth': 6,\n",
    "          'colsample_bytree': 0.7,\n",
    "         }\n",
    "lg = LGBMClassifier(silent=False,  **params)\n",
    "\n",
    "min_child_samples_s = range(40,60,10) \n",
    "tuned_parameters = dict( min_child_samples = min_child_samples_s)\n",
    "\n",
    "grid_search = GridSearchCV(lg, n_jobs=4,  param_grid=tuned_parameters, cv = kfold, scoring=\"roc_auc\", verbose=5, refit = False)\n",
    "grid_search.fit(X_train , y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.845889972267\n",
      "{'min_child_samples': 40}\n"
     ]
    }
   ],
   "source": [
    "# examine the best model\n",
    "print(grid_search.best_score_)\n",
    "print(grid_search.best_params_)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAD8CAYAAABpcuN4AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3Xd0VHX6x/H3k0AooQhSRIpBigKC\ngBGQkqh0VEDXAipWwEaN7lrXVddVV3cJKGDBjgVRRJEOrkvokEgHqSJViKIoIiDw/P6Y8fyycQgD\nJJmUz+scT2a+9zt3nu+JzCf3ztxnzN0RERE5lqhIFyAiInmbgkJERLKkoBARkSwpKEREJEsKChER\nyZKCQkREsqSgEBGRLCkoREQkSwoKERHJUpFIF5AdKlSo4HFxcZEuQ0QkX0lLS/vO3Sseb16BCIq4\nuDhSU1MjXYaISL5iZt+EM0+nnkREJEsKChERyZKCQkREsqSgEBGRLCkoREQkSwoKERHJkoJCRESy\nVKiDYsGm73ltztccOaqvgxUROZZCHRSTlu/k7xNXc/VL81i/6+dIlyMikicV6qB4olsDhl7XmM3f\n/cJlz8/h+c/Xc+jw0UiXJSKSpxTqoDAzujepyoykRDqedwZDZqyj6/A5LN/2Y6RLExHJMwp1UPyu\nQqlivNCzCaNuiueH/YfoPmIuT09ew6+HjkS6NBGRiFNQZNC+fmWmD07kugur83LKJjoPS2HBpu8j\nXZaISEQpKDIpW6IoT1/ViPd6N+eoQ49XFvDw+BX8fOC3SJcmIhIRCopjaFm7AlMHtaF365q8v2gL\nHZJT+M9XuyJdlohIrlNQZKFkTBEeubw+4+5qSeniRbjtzVQGjVnCnl8ORbo0EZFco6AIQ5Ma5ZjY\nvw0D29Zh0oqdtBsyiwnLduCuC/VEpOBTUIQppkgUg9vX5bP+ralergQD3l9Cn7dT+XbvgUiXJiKS\noxQUJ+jcM8rw8d2teLhLPeZs+I72Q2bx/qItOroQkQIrrKAws05mttbMNpjZAyG21zCzL8xsiZkt\nN7MuIbbvM7P7MoxtNrMVZrbUzFIzjDc2swW/j5tZs1NZYE6IjjL6JJzN1IEJNKhahgc/XsH1oxby\nzfe/RLo0EZFsd9ygMLNoYATQGagP9DSz+pmmPQKMdfcmQA9gZKbtycCUELu/xN0bu3t8hrFngcfd\nvTHwaPB+nhRXIZb3erfg6asasnL7XjoOTeHV2ZvUZFBECpRwjiiaARvcfZO7HwLGAN0yzXGgTPB2\nWWDH7xvMrDuwCVgVZk3H3FdeFBVl9GxWgxlJibSuXYEnJ63hqhfnsfZbNRkUkYIhnKCoCmzNcH9b\ncCyjx4AbzWwbMBnoD2BmscD9wOMh9uvAdDNLM7O+GcYHAc+Z2VbgX8CDYdQYcWeULc6om+J5vmcT\ntu7Zz+UvzCZ5xjo1GRSRfC+coLAQY5nPrfQE3nT3akAXYLSZRREIiGR33xdiH63cvSmBU1r3mFlC\ncPwuYLC7VwcGA6+FLMqsb/A9jNT09PQwlpHzzIyu55/JzKREujSswrDP13P5C7NZulVNBkUk/7Lj\nfVrHzC4CHnP3jsH7DwK4+9MZ5qwCOrn71uD9TUALYBxQPTjtNOAo8Ki7D8/0HI8B+9z9X2a2FzjN\n3d3MDNjr7mXIQnx8vKempmY1JSI+X7OLh8evZPfPB7itVU3u7XAOJWKiI12WiAgAZpaW6T3ikMI5\nolgM1DGzmmYWQ+DN6gmZ5mwB2gafuB5QHEh39zbuHufuccBQ4Cl3H25msWZWOjg/FugArAzuaweQ\nGLx9KbA+jBrzpLb1KjM9KYEezWrw6pyv6Tg0hXkbv4t0WSIiJ+S4QeHuh4F+wDRgDYFPN60ysyfM\nrGtw2r1AHzNbBrwP3OJZH6pUBuYE5y8CJrn71OC2PsC/g9ueAvoeYx/5QpniRXnqyoa836cFUQbX\nj1rIgx8v5yc1GRSRfOK4p57yg7x66imzXw8dYejMdYyavYmKpYvxZPeGtK9fOdJliUghlZ2nniSb\nlIiJ5sEu9fjknlaUKxlDn7dT6ffel3y372CkSxMROSYFRQQ0qnYaE/q1Jql9Xaat+pb2Q2bxyZLt\nagMiInmSgiJCYopEMaBtHSYNaMNZp8cy6IOl3P5WKjt+/DXSpYmI/A8FRYTVrVyacXe15K+X12f+\nxu/pkJzCOwu+4ajagIhIHqGgyAOio4zbW9dk2qAEzq9elkc+WUnPUQv4+js1GRSRyFNQ5CE1Ti/J\nO7c359k/NWL1zp/oNDSFl2dt5PARtQERkchRUOQxZsa1F1ZnZlIiCXUr8vSUr7hy5DxW7/gp0qWJ\nSCGloMijKpcpziu9LmDE9U3ZufdXug6fw7+nr+Xg4SORLk1EChkFRR5mZlzWqAozBifS9fwzeeE/\nG7js+TmkffNDpEsTkUJEQZEPlIuNYch1jXnj1gvZf/AwV780j8c/W8X+Q4cjXZqIFAIKinzkknMq\nMT0pkV4tzuKNuZvpkJzCnPVqMigiOUtBkc+UKlaEJ7qdx9g7LqJodBQ3vraQv3y0jL371WRQRHKG\ngiKfalazPFMGtuGui2sx7svttEuexdSV30a6LBEpgBQU+VjxotHc3+lcPrm7FRVKFePOd9K4590v\nSf9ZTQZFJPsoKAqAhtXKMqFfK/7c8RxmrN5FuyGzGJe2TU0GRSRbKCgKiKLRUdxzSW0mD2xD7Uql\nuPfDZdzyxmK2q8mgiJwiBUUBU7tSKT684yIeu6I+izfvocOQWbw9f7OaDIrISVNQFEBRUcYtrQJN\nBpueVY5HP13Fda/MZ2P6vkiXJiL5kIKiAKteviRv39aM565uxNpvf6bzsNmM/O8GflOTQRE5AQqK\nAs7MuCa+OjPvTeTScyrx7NS1dB8xl5Xb90a6NBHJJxQUhUSl0sV5qdcFvHhDU3b9dJBuI+by3LSv\nOPCbmgyKSNYUFIVM54ZVmJmUwJVNqjLii410eX42qZv3RLosEcnDFBSF0GklY/jXNefz9m3NOPjb\nUa55eT6PTVjFLwfVZFBE/iisoDCzTma21sw2mNkDIbbXMLMvzGyJmS03sy4htu8zs/syjG02sxVm\nttTMUjPN7x98vlVm9uzJLk6yllC3ItMHJ3DzRXG8NT/QZHDWuvRIlyUiecxxg8LMooERQGegPtDT\nzOpnmvYIMNbdmwA9gJGZticDU0Ls/hJ3b+zu8Rme7xKgG9DI3RsA/wp3MXLiYosV4bGuDfjwjoso\nVjSKm19fxL1jl/Hj/kORLk1E8ohwjiiaARvcfZO7HwLGEHghz8iBMsHbZYEdv28ws+7AJmBVmDXd\nBTzj7gcB3H13mI+TUxAfV57JA9rQ75LafLJ0O+2GpDBlxc5IlyUieUA4QVEV2Jrh/rbgWEaPATea\n2TZgMtAfwMxigfuBx0Ps14HpZpZmZn0zjNcF2pjZQjObZWYXhirKzPqaWaqZpaan63RJdiheNJr7\nOp7DhH6tqFymGHe9+yV3jk5j908HIl2aiERQOEFhIcYy94PoCbzp7tWALsBoM4siEBDJ7h7qkuBW\n7t6UwCmte8wsITheBCgHtAD+DIw1sz/U4O6vuHu8u8dXrFgxjGVIuBqcWZZP72nF/Z3O5T9rd9Nu\nyCw+TN2qJoMihVQ4QbENqJ7hfjUynFoKuh0YC+Du84HiQAWgOfCsmW0GBgEPmVm/4LwdwZ+7gfEE\nTnH9/nwfe8Ai4GhwX5KLikRHcdfFtZgysA3nnFGaP3+0nJteX8TWPfsjXZqI5LJwgmIxUMfMappZ\nDIE3qydkmrMFaAtgZvUIBEW6u7dx9zh3jwOGAk+5+3AzizWz0sH5sUAHYGVwX58Alwa31QViAH3f\nZ4TUqliKD/pexN+7NeDLb36g49AU3pj7NUfUZFCk0DhuULj7YaAfMA1YQ+DTTavM7Akz6xqcdi/Q\nx8yWAe8Dt3jW5ykqA3OC8xcBk9x9anDb68DZZraSwBvnNx9nX5LDoqKMXhfFMW1wAhfGlefxz1Zz\n7cvz2bD750iXJiK5wArCa3B8fLynpqYef6KcMndn/JLtPDFxNfsPHmFA29rckViLotG6dlMkvzGz\ntIyXJxyL/nXLCTEzrmpajRmDE2nfoDL/mr6OrsPVZFCkIFNQyEmpWLoYI65vysu9LuC7fYEmg89M\nUZNBkYJIQSGnpGODM5g5OJGrm1bjpVkb6TJsNou+VpNBkYJEQSGnrGzJovzz6ka8c3tzDh05yrUv\nz+evn6zk5wO/Rbo0EckGCgrJNq3rVGD64ARua1WTdxZ+Q8fkFL5Yqw4sIvmdgkKyVcmYIjx6RX0+\nurMlscWKcOsbi0n6YCk//KImgyL5lYJCcsQFZ5Vj4oDWDLi0NhOW7aDdkFlMXL5DbUBE8iEFheSY\nYkWiSepwDp/1b82Zp5Wg33tL6Ds6jV1qMiiSrygoJMfVq1KG8Xe35MHO55KyLp12Q2bxweItOroQ\nyScUFJIrikRHcUdiLaYOSqBelTLcP24FN7y6kC3fq8mgSF6noJBcVbNCLGP6tOAfV57H8m176Tg0\nhdfmqMmgSF6moJBcFxVl3ND8LGYkJXBRrdP5+8TV/OnFeazbpSaDInmRgkIipkrZErx2czzDejTm\nm+9/4bLnZ/P85+s5dPhopEsTkQwUFBJRZka3xlWZmZRIp/OqMGTGOroOn8OyrT9GujQRCVJQSJ5w\neqlivNCzCaNuiueH/Ye4cuRcnpq8hl8PqcmgSKQpKCRPaV+/MjOSErnuwuq8krKJzsNSmL/x+0iX\nJVKoKSgkzylTvChPX9WI93o356hDz1ELeGj8Cn5Sk0GRiFBQSJ7VsnYFpg1KoE+bmoxZtIUOQ1L4\nz1e7Il2WSKGjoJA8rURMNA9fVp+P725F2RJFue3NVAaOWcL3+w5GujSRQkNBIflC4+qn8Vn/1gxq\nV4fJK3bSPjmFT5duVxsQkVygoJB8I6ZIFIPa1WVi/zZUL1+SgWOW0vutVHbu/TXSpYkUaAoKyXfO\nOaM0H9/Vkkcuq8fcjd/RYUgK7y3cwlG1ARHJEWEFhZl1MrO1ZrbBzB4Isb2GmX1hZkvMbLmZdQmx\nfZ+Z3ZdhbLOZrTCzpWaWGmKf95mZm1mFk1mYFGzRUUbvNmczbVAC51Uty0PjV3D9qwvY/N0vkS5N\npMA5blCYWTQwAugM1Ad6mln9TNMeAca6exOgBzAy0/ZkYEqI3V/i7o3dPT7Tc1YH2gNbwlqFFFpn\nnR7Le32a88xVDVm1/Sc6DUthVMomNRkUyUbhHFE0Aza4+yZ3PwSMAbplmuNAmeDtssCO3zeYWXdg\nE7DqBOpKBv4S3K9IlsyMHs1qMCMpkda1K/CPyWu4auRc1n6rJoMi2SGcoKgKbM1wf1twLKPHgBvN\nbBswGegPYGaxwP3A4yH268B0M0szs76/D5pZV2C7uy8LdxEiAGeULc6om+J5oWcTtv3wK5e/MJvk\nGes4eFhtQERORThBYSHGMv+l3xN4092rAV2A0WYWRSAgkt19X4h9tHL3pgROad1jZglmVhJ4GHj0\nuEWZ9TWzVDNLTU9PD2MZUhiYGVecfyYzkhK5rGEVhn2+nitemMOSLT9EujSRfCucoNgGVM9wvxoZ\nTi0F3Q6MBXD3+UBxoALQHHjWzDYDg4CHzKxfcN6O4M/dwHgCp7hqATWBZcHHVAO+NLMzMhfl7q+4\ne7y7x1esWDGsxUrhUT42hqE9mvD6LfH8fOAwV704j79PXM3+Q4cjXZpIvhNOUCwG6phZTTOLIfBm\n9YRMc7YAbQHMrB6BoEh39zbuHufuccBQ4Cl3H25msWZWOjg/FugArHT3Fe5eKcNjtgFN3f3bU1+q\nFEaXnluZ6YMTuKF5DV6b8zWdhs5m3obvIl2WSL5y3KBw98NAP2AasIbAp5tWmdkTwfcTAO4F+pjZ\nMuB94BbP+pLZysCc4PxFwCR3n3oqCxE5ltLFi/Jk94aM6duCKIPrX13IA+OWs/dXNRkUCYcVhBYI\n8fHxnpr6h0sxRP7gwG9HSJ65jlEpm6hYuhhPdm9I+/qVI12WSESYWVrmyxNC0ZXZUqgULxrNg53r\n8ck9rShXMoY+b6fS770v+U5NBkWOSUEhhVKjaqcxoV9r7m1fl+mrdtFuyCzGL9mmJoMiISgopNCK\nKRJF/7Z1mDSgNTUrxDL4g2Xc9uZidvyoJoMiGSkopNCrU7k0H93Zkkcvr8+CTXvokJzC6AXfqMmg\nSJCCQoRAk8HbWtdk+uAEGlc/jb9+spIeoxbwtZoMiigoRDKqXr4ko29vxrN/asSanT/RaWgKL83a\nyOEjRyNdmkjEKChEMjEzrr2wOjOTEkmsW5FnpnxF95FzWb3jp0iXJhIRCgqRY6hcpjgv97qAkTc0\n5du9B+g6fA7/nr5WTQal0FFQiGTBzOjSsAozBifStfGZvPCfDVz2/BzSvlGTQSk8FBQiYSgXG8OQ\naxvz5q0X8uuhI1z90jwe/2wVvxxUk0Ep+BQUIifg4nMqMW1wAr1anMUbczfTcWgKs9erzb0UbAoK\nkRNUqlgRnuh2HmPvuIiY6Ch6vbaIv3y0jL371WRQCiYFhchJalazPJMHtuGui2sx7svttEuexdSV\n6ogvBY+CQuQUFC8azf2dzuXTe1pRsVQx7nwnjbvfTWP3zwciXZpItlFQiGSD86qW5dN+rfhzx3OY\nuWY37YekMC5NTQalYFBQiGSTotFR3HNJbSYPaEPtSqW498Nl3PzGYrb9sD/SpYmcEgWFSDarXakU\nH95xEY93bUDq5j10TE7h7fmb1WRQ8i0FhUgOiIoybm4Zx7RBCTQ9qxyPfrqKa1+ez8b0fZEuTeSE\nKShEclD18iV5+7Zm/Oua81m/ex+dh81mxBcb+E1NBiUfUVCI5DAz4+oLqjEjKYF29Srx3LS1dB8x\nl5Xb90a6NJGwKChEckml0sUZecMFvHRjU3b9dJBuI+by7NSvOPCbmgxK3qagEMllnc6rwudJiVzV\npCoj/7uRLs/PJnXznkiXJXJMCgqRCChbsijPXXM+b9/WjIO/HeWal+fzt09Xsk9NBiUPCisozKyT\nma01sw1m9kCI7TXM7AszW2Jmy82sS4jt+8zsvgxjm81shZktNbPUDOPPmdlXwf2MN7PTTmWBInlZ\nQt2KTB+cwM0XxfH2gm/omJzCrHVqMih5y3GDwsyigRFAZ6A+0NPM6mea9ggw1t2bAD2AkZm2JwNT\nQuz+Endv7O7xGcZmAOe5eyNgHfBgWCsRyadiixXhsa4N+OjOiyheNIqbX19E0til/Lj/UKRLEwHC\nO6JoBmxw903ufggYA3TLNMeBMsHbZYEdv28ws+7AJmBVOAW5+3R3//34ewFQLZzHieR3F5xVnkkD\n2tDvktpMWLqDdkNmMXnFzkiXJRJWUFQFtma4vy04ltFjwI1mtg2YDPQHMLNY4H7g8RD7dWC6maWZ\nWd9jPPdthD4SESmQiheN5r6O5/Bpv1acUbY4d7/7JXeOTmP3T2oyKJETTlBYiLHMvQh6Am+6ezWg\nCzDazKIIBESyu4e6HLWVuzclcErrHjNL+J8nNXsYOAy8G7Ios75mlmpmqenpOqcrBUuDM8vyyd2t\nuL/Tufxn7W7aDZnF2NStajIoERFOUGwDqme4X40Mp5aCbgfGArj7fKA4UAFoDjxrZpuBQcBDZtYv\nOG9H8OduYDyBU1wAmNnNwOXADX6Mfxnu/oq7x7t7fMWKFcNYhkj+UiQ6irsursXUgW0494wy/OWj\n5dz0+iK27lGTQcld4QTFYqCOmdU0sxgCb1ZPyDRnC9AWwMzqEQiKdHdv4+5x7h4HDAWecvfhZhZr\nZqWD82OBDsDK4P1OBE5XdXV3/YuQQu/siqUY07cFf+9+Hl9+8wMdklN4Y+7XHFGTQcklxw2K4BvL\n/YBpwBoCn25aZWZPmFnX4LR7gT5mtgx4H7jlWEcCQZWBOcH5i4BJ7j41uG04UBqYEfzo7EsntTKR\nAiQqyujV4iymJyXS/OzyPP7Zaq55aR4bdv8c6dKkELCCcM4zPj7eU1NTjz9RpABwdz5Zup3HP1vN\n/oNHGNC2Nnck1qJotK6flRNjZmmZLk8ISf9nieQzZsaVTaoxMymR9g0q86/p67jihTms2KYmg5Iz\nFBQi+VSFUsUYcX1TXu51AXt+OUT3kXN5ZoqaDEr2U1CI5HMdG5zBjKRErm5ajZdmbaTzsNks3PR9\npMuSAkRBIVIAlC1RlH9e3Yh3ezfn8NGjXPfKAh75ZAU/H/gt0qVJAaCgEClAWtWuwLRBCdzeuibv\nLtxCx+QUvvhqd6TLknxOQSFSwJSMKcJfL6/PuLtaElusCLe+uZjBHyxlzy9qMignR0EhUkA1rVGO\niQNaM6BtHT5btoP2Q2YxcfkOtQGRE6agECnAihWJJql9XT7r35qq5UrQ770l9B2dxi41GZQToKAQ\nKQTqVSnDx3e15KEu55KyLp12Q2YxZtEWHV1IWBQUIoVEkego+ibUYtqgBOpXKcMDH6/ghlcXsuV7\ntVSTrCkoRAqZuAqxvN+nBU9d2ZDl2/bSYegsXp29SU0G5ZgUFCKFUFSUcX3zGsxISqBlrQo8OWkN\nf3pxHut2qcmg/JGCQqQQq1K2BK/dHM+wHo3Zsmc/lz0/m2Ez13Po8NFIlyZ5iIJCpJAzM7o1rsqM\nwQl0Pq8KyTMDTQaXbf0x0qVJHqGgEBEATi9VjOd7NuHVm+LZ++tvXDlyLv+YtJpfD6nJYGGnoBCR\n/9GufmWmJyXQo1kNRs3+mk7DUpi/UU0GCzMFhYj8QZniRXnqyoa816c5AD1HLeDBj1fwk5oMFkoK\nChE5ppa1KjB1YAJ9E87mg8Vb6DAkhc/X7Ip0WZLLFBQikqUSMdE81KUeH9/dirIlinL7W6kMeH8J\n3+87GOnSJJcoKEQkLI2rn8Zn/VszuF1dpqzcSfvkFD5dul1tQAoBBYWIhC2mSBQD29Vh0oA21Chf\nkoFjltL7rVR27v010qVJDlJQiMgJq1u5NOPuaskjl9Vj7sbvaD8khXcXfsNRtQEpkBQUInJSoqOM\n3m3OZvqgRBpVK8vD41dy/asL2PzdL5EuTbJZWEFhZp3MbK2ZbTCzB0Jsr2FmX5jZEjNbbmZdQmzf\nZ2b3ZRjbbGYrzGypmaVmGC9vZjPMbH3wZ7lTWaCI5Kwap5fk3d7Neeaqhqza/hMdh6bwSspGDh9R\nG5CC4rhBYWbRwAigM1Af6Glm9TNNewQY6+5NgB7AyEzbk4EpIXZ/ibs3dvf4DGMPAJ+7ex3g8+B9\nEcnDzIwezWowIymRNnUq8tTkr/jTi/P46tufIl2aZINwjiiaARvcfZO7HwLGAN0yzXGgTPB2WWDH\n7xvMrDuwCVgVZk3dgLeCt98Cuof5OBGJsDPKFmfUTRcw/PombPvhVy5/fg5DZqzj4GG1AcnPwgmK\nqsDWDPe3Bccyegy40cy2AZOB/gBmFgvcDzweYr8OTDezNDPrm2G8srvvBAj+rBSqKDPra2apZpaa\nnp4exjJEJDeYGZc3OpOZSYlccf6ZPP/5ei5/fg5fbvkh0qXJSQonKCzEWOaPNvQE3nT3akAXYLSZ\nRREIiGR33xdiH63cvSmBU1r3mFnCCdSNu7/i7vHuHl+xYsUTeaiI5IJysTEkX9eYN265kH0HD/On\nF+fx94mr2X/ocKRLkxMUTlBsA6pnuF+NDKeWgm4HxgK4+3ygOFABaA48a2abgUHAQ2bWLzhvR/Dn\nbmA8gVNcALvMrApA8OfuE16ViOQZl5xbiemDE7iheQ1em/M1HYemMHfDd5EuS05AOEGxGKhjZjXN\nLIbAm9UTMs3ZArQFMLN6BIIi3d3buHucu8cBQ4Gn3H24mcWaWeng/FigA7AyuK8JwM3B2zcDn570\n6kQkTyhdvChPdm/IB31bUCQqihteXcgD45az91c1GcwPjhsU7n4Y6AdMA9YQ+HTTKjN7wsy6Bqfd\nC/Qxs2XA+8AtnvV1/ZWBOcH5i4BJ7j41uO0ZoL2ZrQfaB++LSAHQ/OzTmTKwDXckns3Y1K20HzKL\n6au+jXRZchxWEPq0xMfHe2pq6vEnikiesXzbj/zlo+V89e3PXNaoCo9d0YCKpYtFuqxCxczSMl2e\nEJKuzBaRiGhULdBk8L4OdZmxahftk2cxfsk2NRnMgxQUIhIxRaOj6HdpHSYPbM3ZFWIZ/MEybn1z\nMdt/VJPBvERBISIRV7tSaT68syV/u6I+CzftocOQWYxeoCaDeYWCQkTyhOgo49ZWNZk+OIEmNcrx\n109W0uOVBWxKD3UZluQmBYWI5CnVy5dk9O3NePbqRnz17U90Hjabl2apyWAkKShEJM8xM66Nr87M\npEQuPqciz0z5iu4j57J6h5oMRoKCQkTyrEplivNyr3hevKEp3+49SNfhc/jXtLUc+E1NBnOTgkJE\n8rzODaswMymBbo2rMvyLDVz2/GzSvtkT6bIKDQWFiOQLp5WM4d/Xns9btzXjwG9Hufql+Tw2YRW/\nHFSTwZymoBCRfCWxbkWmDU7gphZn8ea8zXQcmsLs9fqqgZykoBCRfKdUsSI83u08PrzzImKKRNHr\ntUX8+cNl7N2vJoM5QUEhIvnWhXHlmTygDXdfXIuPl2ynXfIspq7cGemyChwFhYjka8WLRvOXTufy\n6T2tqFiqGHe+8yV3vZPG7p8PRLq0AkNBISIFwnlVy/Jpv1b8ueM5fP7VbtoPSeGjNDUZzA4KChEp\nMIpGR3HPJbWZPKANdSqV4r4Pl3HzG4vZ9sP+SJeWrykoRKTAqV2pFGPvuIgnujUgbfMeOiSn8Na8\nzWoyeJIUFCJSIEVFGTddFMe0wQnEx5XnbxNWce3L89mwW00GT5SCQkQKtGrlSvLWrRfy72vOZ/3u\nfXQZNpsRX2zgNzUZDJuCQkQKPDPjTxdUY2ZSIu3qV+K5aWvpNnwuK7fvjXRp+YKCQkQKjYqlizHy\nhgt46campO87SLcRc/nn1K/UZPA4FBQiUuh0Oq8KMwcnclWTqrz43410GTabxZvVZPBYFBQiUiiV\nLVmU5645n9G3N+PQkaNc89J8Hv10JfvUZPAPwgoKM+tkZmvNbIOZPRBiew0z+8LMlpjZcjPrEmL7\nPjO7L9N4dPAxEzOMtTWzL82ojga5AAAK8klEQVRsqZnNMbPaJ7s4EZHjaVOnItMGJXBrqzhGL/iG\njskp/Hft7kiXlaccNyjMLBoYAXQG6gM9zax+pmmPAGPdvQnQAxiZaXsyMCXE7gcCazKNvQjc4O6N\ngfeC+xYRyTGxxYrwtysa8NGdLSkRE80tbywmaexSfvjlUKRLyxPCOaJoBmxw903ufggYA3TLNMeB\nMsHbZYEdv28ws+7AJmBVxgeYWTXgMuDVcPclIpKTLjirHJMGtKb/pbWZsHQH7ZNnMXnFzkLfBiSc\noKgKbM1wf1twLKPHgBvNbBswGegPYGaxwP3A4yH2OxT4C5D5w8y9gcnBffUCngmjRhGRbFGsSDT3\ndjiHCf1aU6VsCe5+90vufCeN3T8V3iaD4QSFhRjLHK89gTfdvRrQBRhtZlEEAiLZ3f/nUkgzuxzY\n7e5pIfY9GOgS3NcbwJCQRZn1NbNUM0tNT9eXlohI9qp/ZhnG392SBzqfy3/XptNuyCzGpm4tlEcX\ndrxFm9lFwGPu3jF4/0EAd386w5xVQCd33xq8vwloAYwDqgennUbg6OFRAkckvYDDQHECp5o+JhAS\nC9y9VnA/NYCp7p75PZH/ER8f76mpqeGvWkTkBGxK38cDH69g0dd7aF27Ak9f1ZDq5UtGuqxTZmZp\n7h5/vHnhHFEsBuqYWU0ziyHwZvWETHO2AG2DT1yPwIt/uru3cfc4d48jcKrpKXcf7u4Punu14HgP\n4D/ufiPwA1DWzOoG99ueP77ZLSKSq86uWIoxfVrwZPfzWLr1Rzokp/D6nK85UkiaDB43KNz9MNAP\nmEbgRXusu68ysyfMrGtw2r1AHzNbBrwP3OIncXwWfK4+wLjgvnoBfz7R/YiIZLeoKOPGFmcxfXAC\nzc8uzxMTV3PNS/NYv+vnSJeW44576ik/0KknEclN7s6nS3fw+Ger+OXgEfpfWps7L65F0ej8dQ1z\ndp56EhGRDMyM7k2qMiMpkQ4NKvPvGeu44oU5rNhWMJsMKihERE5ShVLFGH59U17pdQE/7D9EtxFz\neHrKmgLXZFBBISJyijo0OIPpgxO57sLqvDxrE52HzWbBpu8jXVa2UVCIiGSDsiWK8vRVjXivd3OO\nHHV6vLKAh8ev4OcDv0W6tFOmoBARyUYta1dg6qA29G5dk/cXbaFDcgpffJW/mwwqKEREslnJmCI8\ncnl9xt3VklLFinDrm4sZNGYJe/Jpk0EFhYhIDmlSoxwTB7RmYNs6TFy+k/ZDZvHZsh35rg2IgkJE\nJAcVKxLN4PZ1mTigNdXKlaD/+0vo83Ya3+7NP00GFRQiIrng3DPK8PHdrXi4Sz3mbEin/ZBZvL9o\nS744ulBQiIjkkugoo0/C2UwdmECDqmV48OMVXD9qId98/0ukS8uSgkJEJJfFVYjlvd4teOrKhqzc\nvpeOQ1N4dfamPNtkUEEhIhIBUVHG9c1rMD0pgVa1KvDkpDVc9eI81n6b95oMKihERCKoStkSvHpz\nPM/3bMLWPfu5/IXZDJ25jkOHM3/5Z+QoKEREIszM6Hr+mcxMSqRLwyoMnbmeK16Yw9KtP0a6NEBB\nISKSZ5SPjWFYjya8dnM8e3/9jatGzuUfk1bz66HINhlUUIiI5DFt61VmelICPZrVYNTsr+k4NIV5\nG7+LWD0KChGRPKhM8aI8dWVD3u/TAjO4ftRCHvx4BT9FoMmggkJEJA+7qNbpTB2YQN+Es/lg8Rba\nD5nFzNW7crUGBYWISB5XIiaah7rUY/zdrShXMobeb6cy4P0lfL/vYK48v4JCRCSfOL/6aUzo15qk\n9nWZsnIn7YbMYv7GnP+CJAWFiEg+ElMkigFt6zBpQBvOq1qWuAolc/w5i+T4M4iISLarW7k0o29v\nnivPpSMKERHJUlhBYWadzGytmW0wswdCbK9hZl+Y2RIzW25mXUJs32dm92Uajw4+ZmKGMTOzf5jZ\nOjNbY2YDTnZxIiJy6o576snMooERQHtgG7DYzCa4++oM0x4Bxrr7i2ZWH5gMxGXYngxMCbH7gcAa\noEyGsVuA6sC57n7UzCqFvxwREclu4RxRNAM2uPsmdz8EjAG6ZZrj/P+LfVlgx+8bzKw7sAlYlfEB\nZlYNuAx4NdO+7gKecPejAO6ev7+VXEQknwsnKKoCWzPc3xYcy+gx4EYz20bgaKI/gJnFAvcDj4fY\n71DgL0DmFom1gOvMLNXMpphZnTBqFBGRHBJOUFiIsczfrtETeNPdqwFdgNFmFkUgIJLdfd//7NDs\ncmC3u6eF2Hcx4IC7xwOjgNdDFmXWNxgmqenp6WEsQ0RETkY4H4/dRuA9g99VI8OppaDbgU4A7j7f\nzIoDFYDmwNVm9ixwGnDUzA4QOCLpGnzTuzhQxszecfcbg883Lrjf8cAboYpy91eAVwDi4+Pz5tdC\niYgUAOEcUSwG6phZTTOLAXoAEzLN2QK0BTCzegRe/NPdvY27x7l7HIFTTU+5+3B3f9DdqwXHewD/\nCYYEwCfApcHbicC6k1+eiIicquMeUbj7YTPrB0wDooHX3X2VmT0BpLr7BOBeYJSZDSZwWuoWdz/Z\nv/KfAd4N7msf0Pt4D0hLS/vOzL45yeerAESuf29kaM2Fg9ZcOJzKms8KZ5Kd/Ot5wWBmqcH3QwoN\nrblw0JoLh9xYs67MFhGRLCkoREQkSwqK4CenChmtuXDQmguHHF9zoX+PQkREsqYjChERyVKhC4rM\nHWuD14csNLP1ZvZB8FqRAiXEmt8NdgNeaWavm1nRSNeY3UJ1Jg6Ov2Bm+471uPwsxO+5wHdiDrHm\ntmb2pZktNbM5ZlY70jVmJzPbbGYrgutLDY6VN7MZwdewGWZWLruft9AFBf/fsfZ3/yTQZqQO8AOB\nq8wLmsxrfhc4F2gIlCCMa1XyocxrxsziCXQIKKgyr/kW/r8Tcz0CDT0LmsxrfhG4wd0bA+8R6Gxd\n0Fzi7o0zfCT2AeDz4GvY58H72apQBUXmjrVmZgSuAv8oOOUtoHtkqssZobr0uvtkDwIWEWjLUmCE\nWnOwXf5zBBpRFjjH6MZcoDsxH2PNx+xkXYB1I/DaBTn0GlaogoI/dqw9HfjR3Q8H74fqjJvfHatL\nL8FTTr2AqbldVA4LteZ+wAR33xmZknJcqDUX9E7ModbcG5gc7GTdi0Cnh4LEgelmlmZmfYNjlX//\n/zr4M9u/w6fQBMUxOtaG0xk33zpOl16AkUCKu8/OxbJyVKg1m9mZwDXACxErLAdl8XsOqxNzfpTF\nmgcDXYKdrN8AhuR6cTmrlbs3BToD95hZQm48aTjdYwuKVmTqWEvgL5LTzKxI8KgiVGfc/OwPa/69\nS6+Z/Q2oCNwR0QqzX6jf8yrgILAhcLaRkma2wd0LyhudIX/PhNmJOZ8KteZJBN6PWRic8wEF7GjZ\n3XcEf+42s/EEvlhul5lVcfedZlYFyP5TjO5e6P4DLgYmBm9/CPQI3n4JuDvS9eXCmnsD84ASka4r\nt9acaXxfpGvLpd/zM8BtGcYXR7q+nFwzgT98vwPqBsdvB8ZFur5sXGcsUDrD7XkEvt7hOeCB4PgD\nwLPZ/dyF6YjiWO4HxpjZk8AS4LUI15MbXgK+AeYH/8L+2N2fiGxJkgNOuBNzfuaBTtd9gHFmdpTA\npxhvi3BZ2akyMD74b7YI8J67TzWzxcBYM7udwFc+XJPdT6wrs0VEJEuF5s1sERE5OQoKERHJkoJC\nRESypKAQEZEsKShERCRLCgoREcmSgkJERLKkoBARkSz9H4FfdIeH2P3uAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1a0b86ea90>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# plot CV误差曲线\n",
    "test_means = grid_search.cv_results_[ 'mean_test_score' ]\n",
    "test_stds = grid_search.cv_results_[ 'std_test_score' ]\n",
    "train_means = grid_search.cv_results_[ 'mean_train_score' ]\n",
    "train_stds = grid_search.cv_results_[ 'std_train_score' ]\n",
    "\n",
    "x_axis = min_child_samples_s\n",
    "\n",
    "plt.plot(x_axis, test_means)\n",
    "#plt.errorbar(x_axis, -test_scores, yerr=test_stds ,label = ' Test')\n",
    "#plt.errorbar(x_axis, -train_scores, yerr=train_stds,label =  +' Train')\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### min_child_samples=40"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 列采样参数 sub_feature/feature_fraction/colsample_bytree"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fitting 5 folds for each of 5 candidates, totalling 25 fits\n",
      "[CV] colsample_bytree=0.5 ............................................\n",
      "[CV] colsample_bytree=0.5 ............................................\n",
      "[CV] colsample_bytree=0.5 ............................................\n",
      "[CV] colsample_bytree=0.5 ............................................\n",
      "[CV] ....... colsample_bytree=0.5, score=0.861714971703, total=   0.9s\n",
      "[CV] colsample_bytree=0.5 ............................................\n",
      "[CV] ....... colsample_bytree=0.5, score=0.831381810407, total=   0.9s\n",
      "[CV] ....... colsample_bytree=0.5, score=0.836188303893, total=   0.9s\n",
      "[CV] colsample_bytree=0.6 ............................................\n",
      "[CV] ....... colsample_bytree=0.5, score=0.850134164523, total=   0.9s\n",
      "[CV] colsample_bytree=0.6 ............................................\n",
      "[CV] colsample_bytree=0.6 ............................................\n",
      "[CV] ....... colsample_bytree=0.5, score=0.851513115937, total=   0.9s\n",
      "[CV] colsample_bytree=0.6 ............................................\n",
      "[CV] ........ colsample_bytree=0.6, score=0.86067558452, total=   0.9s\n",
      "[CV] ....... colsample_bytree=0.6, score=0.836473675184, total=   0.8s\n",
      "[CV] colsample_bytree=0.6 ............................................\n",
      "[CV] ....... colsample_bytree=0.6, score=0.833753868916, total=   0.8s\n",
      "[CV] colsample_bytree=0.7 ............................................\n",
      "[CV] colsample_bytree=0.7 ............................................\n",
      "[CV] ........ colsample_bytree=0.6, score=0.84834656689, total=   0.8s\n",
      "[CV] colsample_bytree=0.7 ............................................\n",
      "[CV] ....... colsample_bytree=0.6, score=0.849594889259, total=   0.9s\n",
      "[CV] colsample_bytree=0.7 ............................................\n",
      "[CV] ....... colsample_bytree=0.7, score=0.862881495455, total=   0.9s\n",
      "[CV] colsample_bytree=0.7 ............................................\n",
      "[CV] ....... colsample_bytree=0.7, score=0.835298576573, total=   1.0s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=4)]: Done  10 tasks      | elapsed:    3.9s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV] colsample_bytree=0.8 ............................................\n",
      "[CV] ....... colsample_bytree=0.7, score=0.848797569978, total=   0.9s\n",
      "[CV] ........ colsample_bytree=0.7, score=0.84826059048, total=   0.8s\n",
      "[CV] colsample_bytree=0.8 ............................................\n",
      "[CV] ....... colsample_bytree=0.7, score=0.834211564386, total=   1.0s\n",
      "[CV] colsample_bytree=0.8 ............................................\n",
      "[CV] colsample_bytree=0.8 ............................................\n",
      "[CV] ....... colsample_bytree=0.8, score=0.860640370434, total=   1.1s\n",
      "[CV] colsample_bytree=0.8 ............................................\n",
      "[CV] ....... colsample_bytree=0.8, score=0.831301202037, total=   0.9s\n",
      "[CV] ........ colsample_bytree=0.8, score=0.83630343566, total=   1.0s\n",
      "[CV] colsample_bytree=0.9 ............................................\n",
      "[CV] ....... colsample_bytree=0.8, score=0.843657351523, total=   0.9s\n",
      "[CV] colsample_bytree=0.9 ............................................\n",
      "[CV] colsample_bytree=0.9 ............................................\n",
      "[CV] ....... colsample_bytree=0.8, score=0.849222401343, total=   0.9s\n",
      "[CV] colsample_bytree=0.9 ............................................\n",
      "[CV] ....... colsample_bytree=0.9, score=0.864116846739, total=   0.9s\n",
      "[CV] ....... colsample_bytree=0.9, score=0.831419656322, total=   0.9s\n",
      "[CV] ....... colsample_bytree=0.9, score=0.835789401475, total=   0.9s\n",
      "[CV] colsample_bytree=0.9 ............................................\n",
      "[CV] ....... colsample_bytree=0.9, score=0.843563569511, total=   1.0s\n",
      "[CV] ....... colsample_bytree=0.9, score=0.849897471123, total=   0.4s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=4)]: Done  25 out of  25 | elapsed:    8.3s finished\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "GridSearchCV(cv=StratifiedKFold(n_splits=5, random_state=3, shuffle=True),\n",
       "       error_score='raise',\n",
       "       estimator=LGBMClassifier(boosting_type='goss',\n",
       "        categorical_feature=[0, 1, 3, 5, 6, 12, 15, 16, 17, 18, 19, 20],\n",
       "        colsample_bytree=1.0, is_unbalance=True, learning_rate=0.1,\n",
       "        max_bin=255, max_depth=6, min_child_samples=40, min_child_weight=5,\n",
       "        min_split_gain=0.0, n_estimators=38, n_jobs=4, num_leaves=70,\n",
       "        objective='binary', random_state=0, reg_alpha=0.0, reg_lambda=0.0,\n",
       "        silent=False, subsample=1.0, subsample_for_bin=50000,\n",
       "        subsample_freq=1),\n",
       "       fit_params=None, iid=True, n_jobs=4,\n",
       "       param_grid={'colsample_bytree': [0.5, 0.6, 0.7, 0.8, 0.9]},\n",
       "       pre_dispatch='2*n_jobs', refit=False, return_train_score='warn',\n",
       "       scoring='roc_auc', verbose=5)"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "params = {'boosting_type': 'goss',\n",
    "          'objective': 'binary',\n",
    "          'is_unbalance':True,\n",
    "          'categorical_feature': [0,1,3,5,6,12,15,16,17,18,19,20],\n",
    "          'n_jobs': 4,\n",
    "          'learning_rate': 0.1,\n",
    "          'n_estimators':n_estimators_1,\n",
    "          'num_leaves': 70,\n",
    "          'max_depth': 7,\n",
    "          'min_child_samples':40\n",
    "          #'colsample_bytree': 0.7,\n",
    "         }\n",
    "lg = LGBMClassifier(silent=False,  **params)\n",
    "\n",
    "colsample_bytree_s = [i/10.0 for i in range(5,10)]\n",
    "tuned_parameters = dict( colsample_bytree = colsample_bytree_s)\n",
    "\n",
    "grid_search = GridSearchCV(lg, n_jobs=4,  param_grid=tuned_parameters, cv = kfold, scoring=\"roc_auc\", verbose=5, refit = False)\n",
    "grid_search.fit(X_train , y_train)\n",
    "#grid_search.best_estimator_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.846186430268\n",
      "{'colsample_bytree': 0.5}\n"
     ]
    }
   ],
   "source": [
    "# examine the best model\n",
    "print(grid_search.best_score_)\n",
    "print(grid_search.best_params_)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZAAAAD8CAYAAABZ/vJZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3Xl8VOXZ//HPlYQkEAhbwhpCEnbc\nWCKLkIjiAqhA1SpYVJDFtmJrq1Zcnj7or7V9rC3aurQBFUQFqa0tKopUUVDZEjZZREKAEEAIO2HJ\nev3+mBMcpgOZhCRnklzv1yuvmbnPfc5cM8Z8Ofc55z6iqhhjjDHlFeJ2AcYYY2omCxBjjDEVYgFi\njDGmQixAjDHGVIgFiDHGmAqxADHGGFMhFiDGGGMqxALEGGNMhViAGGOMqZCwQDqJyBDgeSAUmKGq\nv/dZHg/MApo4faao6gKf5ZuAqar6rNPWBJgBXAwocI+qLhORPwA3AQXANmCcqh4RkQRgM7DF2exy\nVf3x+eqOiYnRhISEQD6iMcYYR0ZGxgFVjS2rX5kBIiKhwIvAtUAOsEpE5qvqJq9uTwDzVPVlEekO\nLAASvJZPAz702fTzwEeqequIhAMNnPZFwKOqWiQi/wc8CjziLNumqj3KqrlUQkIC6enpgXY3xhgD\niMjOQPoFMoTVB8hU1SxVLQDmAiN8+igQ7TxvDOzxKmQkkAVs9GqLBlKBVwBUtUBVjzjPP1bVIqfr\nciAukA9ijDGmegUSIG2BXV6vc5w2b1OBMSKSg2fv434AEYnCs/fwpE//JCAXeE1E1ojIDKevr3s4\ne88l0en/uYikBFC7McaYKhJIgIifNt8pfEcDM1U1DhgGzBaREDzBMU1V83z6hwG9gJdVtSdwAphy\n1puKPA4UAW86TXuBeKf/L4G3nD0ZfNabJCLpIpKem5sbwMczxhhTEYEcRM8B2nm9jsNriMoxHhgC\n4BwIjwRigL7ArSLyDJ4D7CUichp4B8hR1RXO+u/gFSAicjdwIzBYnfnmVTUfyHeeZ4jINqAzcNZB\nDlVNA9IAkpOTba56Y4ypIoEEyCqgk4gkAruBUcAdPn2ygcHATBHpBkQCuap6ZphJRKYCear6gvN6\nl4h0UdUtzrqbnPYheIa9rlTVk17rxwKHVLVYRJKATniOrRhjjHFBmQHinA01GViI5xTdV1V1o4g8\nBaSr6nzgQWC6iPwCz/DW2NI9h/O4H3jTOQMrCxjntL8ARACLRAS+P103FXhKRIqAYuDHqnqonJ/X\nGGNMJZHafEfC5ORktdN4jTGmfEQkQ1WTy+pnV6L7cbqwmKnzN5Jz+GTZnY0xpo6yAPFj3a4jzFmZ\nzdV//JzffbiZo6cK3S7JGGOCjgWIH32TmrP4oUHceGlr0pZkMegPi3nty+0UFJW4XZoxxgQNC5Bz\naNOkPn+6rQfvTR5I9zbRPPneJq6b9jkffr2X2nzcyBhjAmUBUoaL2zbmjfF9eW3c5dQLDeEnb67m\n1r8uY3X2YbdLM8YYV1mABEBEuKpLCz78eQq/u/kSsg+d5OaXvuK+N1ez8+AJt8szxhhX2Gm8FXAi\nv4i0JVmkLcmiqKSEO/slcP/VHWkaFV7p72WMMdUt0NN4LUAuwL5jp5m26Fvmpe+iYUQYk6/uyF39\nE4isF1pl72mMMVXNrgOpBi2jI/n9LZfy4c9T6dW+KU8v+IZr/vQ5/167m5KS2hvMxhgDFiCVokur\nRswc14c3xvelUWQ9fj53LT946UtWZB10uzRjjKkyFiCVaGCnGN6/fyDP/vAy9h3L5/a05Ux8PZ1t\nub6z2RtjTM1nx0CqyKmCYl79cjsvf7aNU4XF3NEnnp9f04mYhhGu1GOMMYGyg+gEx2SKB/Lyef4/\nW3lrZTb164Xyk0EdGD8w0Q60G2OClh1EDxIxDSP4fyMvZuEDqfTv0Jw/LNzCVc9+xj8ycuxAuzGm\nRrMAqSYdWzRk+l3JvD2pHy0aRfDg39dx41++4MvMA26XZowxFWIBUs36JjXn3Z8O4PlRPTh6qpAf\nzVjB2NdWsuW7426XZowx5RJQgIjIEBHZIiKZIjLFz/J4EVksImtEZL2IDPOzPE9EHvJqayIi74jI\nNyKyWUT6O+3NRGSRiGx1Hps67SIif3ZqWC8ivS7so7snJEQY0aMtnzx4JY8N60rGzsMMfX4Jj/5z\nPfuPnXa7PGOMCUiZASIiocCLwFCgOzBaRLr7dHsCmKeqPfHcM/0ln+XTgA992p4HPlLVrsBlwGan\nfQrwiap2Aj5xXuO8fyfnZxLwcpmfLshF1gtlUmoHljx8FWOvSOSdjBwGPfsZz/3nW07kF7ldnjHG\nnFcgeyB9gExVzVLVAmAuMMKnjwLRzvPGwJ7SBSIyEs89zzd6tUXjucf5KwCqWqCqR5zFI4BZzvNZ\nwEiv9tfVYznQRERaB/Qpg1zTqHB+fVN3Fv3iSgZ1ieW5/2xl0LOfMXdlNsV2oN0YE6QCCZC2wC6v\n1zlOm7epwBgRyQEWAPcDiEgU8AjwpE//JCAXeM0Z9prh9AVoqap7AZzHFuWoo0ZLiInipR/15h8/\n6U+7pvWZ8s+vGfr8EhZv2W/3IDHGBJ1AAkT8tPn+NRsNzFTVOGAYMFtEQvAExzRV9b0UOwzoBbzs\nDHud4PuhqgupAxGZJCLpIpKem5tbxiaDU+/2zfjHT67g5R/1Ir+ohHGvrWLMKyvYuOeo26UZY8wZ\ngQRIDtDO63UcXkNUjvHAPABVXQZEAjFAX+AZEdkBPAA8JiKTnW3mqOoKZ/138AQKwL7SoSnncX85\n6kBV01Q1WVWTY2NjA/h4wUlEGHpJaxb94kr+96bubNxzjBv/8gW/nLeWPUdOuV2eMcYEFCCrgE4i\nkigi4XgOks/36ZMNDAYQkW54AiRXVVNUNUFVE4DngKdV9QVV/Q7YJSJdnPUHA5uc5/OBu53ndwP/\n9mq/yzkbqx9wtHSoqzYLDwth3IBEPn/4KialJvH++r1c9exn/GHhNxw/Xeh2ecaYOqzMAFHVImAy\nsBDPmVLzVHWjiDwlIsOdbg8CE0VkHTAHGKtlD9rfD7wpIuuBHsDTTvvvgWtFZCtwrfMaPMdWsoBM\nYDrw0wA/Y63QuH49Hh3ajU8fvJKhF7fixcXbGPSHz5i9bAeFxSVul2eMqYNsLqwaan3OEX77wWZW\nbD9EUmwUU4Z05druLRHxd6jIGGMCZ3Nh1XKXxjVh7qR+zLgrGQEmzc7g9rTlrNt1pMx1jTGmMliA\n1GAiwjXdW7LwgVR+M/JisnLzGPHil/xszhp2HTrpdnnGmFrOhrBqkbz8Iv72+TamL82ipATGDkjg\nvkEdadygntulGWNqEBvCqoMaRoTx4HVdWPzQIEb0aMP0pVlc+exiXvliOwVFdqDdGFO5LEBqodaN\n6/OHH17GB/encHGbxvy/9zdx7bTP+WD9Xrui3RhTaWwIq5ZTVT7/NpffLfiGLfuO0yu+CY/f0I3e\n7Zu5XVqdUlKi7D5yisz9eWTuzyP70EnG9GtPl1aN3C7NmP8S6BBWWHUUY9wjIgzq0oKUTrG8k7GL\nP378Lbe8vIyhF7fikSFdSYiJKnsjJmAFRSXsPHjiTFBk5noet+Xmcbrw7GHE7EMnmXVPH5cqNebC\nWYDUEaEhwu2Xx3PTZW2YvmQ7f1uyjUWb9jGmX3t+NrgTzaLC3S6xRjmRX8Q2Jxy8w2LnwZNnzaDc\ntkl9OrRoSN/E5nRs0fDMz5yV2fxh4Ra2fHfc9kJMjWVDWHXU/uOnmbZoK2+vyiYqIoz7rurI2CsS\niKwX6nZpQeVgXv5ZexKZ+/PYtj+PPUe/v/FXWIjQvnmDswKiY2wjkmKjiIrw/2+0IycL6P+7Txl2\nSWv+eNtl1fVxjAlIoENYFiB13Lf7jvP7D7/h02/207ZJfR6+vgvDL2tDSEjduaJdVdlz9LTX3sTx\nM88Pn/x+vrHIeiF0iPUERCevsIhvFkV4WPnPR5k6fyNvrtjJ0l9dTavGkZX5kYy5IBYgWICUx1eZ\nB/jtgs1s3HOMS9o25rFh3ejfobnbZVWqwuISdh48eeaYxJk9itw8ThYUn+nXpEE9OsZ+HxAdWjSk\nY2xD2japX6nBmn3wJIOeXcyk1A5MGdq10rZrzIWyAMECpLxKSpR/rd3Nswu3sOfoaa7p1oIpQ7vS\nsUXNGqM/VVB8juMTJygs/v73vXXjSE9AeIVFxxYNaR4VXm1zit335mqWbM1l2aODaXiO4S5jqpsF\nCBYgFXW6sJhXv9zOy4u3cbKwmFGXt+OBazoT2yjC7dLOcuRkAZn789i6/+yw2O11v5QQgfbNo7yO\nTXy/VxEMf7DXZB/mBy99xf/c2J3xAxPdLscYwAIEsAC5UAfz8vnzJ1t5c0U2EWEh/GRQB8YPTKJ+\nePUdaFdVvjt2+uy9CWfY6UBewZl+EWEhJMWeHRIdWzQkIaYBEWHBfWLAbX9dxu4jp/j84UGEhdq1\nvcZ9FiBYgFSWrNw8/u+jb1i4cR+toiN58LrO3NwrjtBKPB5QVFxC9qGTZ187sT+PbbknyMsvOtMv\nOjLs7LOdnDOe2jatX6n1VKf/bNrHhNfT+fPongy/rI3b5RhjAQIWIJVt1Y5D/OaDzazbdYSurRrx\n+A3dSOlUvtsGny4sJiv3BFv3H2ebV1jsOHCSAq8bY7WMjvivIaeOLRoS2zCi1t3zpKREuWba5zQI\nD+W9yQNr3eczNY8FCBYgVUFVeX/9Xp5Z+A27Dp0itXMsjw3rStdW0Wf1O3qq8MxehPc1FLsOn6T0\nVy5EoF2zBmeFRCfnMTqybs0gPGdlNo/+82vmTOxX685+MzVPpQaIiAwBngdCgRmq+nuf5fHALKCJ\n02eKqi7wWb4JmKqqzzptO4DjQDFQVFqsiLwNlN4rvQlwRFV7iEgCnlvqbnGWLVfVH5+vbguQqpNf\nVMzsZTv58ydbycsvYmSPtkRFhJ0Zgso9nn+mb3hYCEkxUWdOhy0dekqMibILFx2nC4sZ+H+fcmlc\nE14de7nb5Zg6rtLmwhKRUOBFPPcnzwFWich8Vd3k1e0JPPdKf1lEuuO5f3mC1/JpwId+Nn+Vqh7w\nblDV273e+4/AUa/F21S1R1k1m6oXERbKhJQkbu0dxwufZvL6sp1EhIXQoUVDruwce9bwU7tmDWrs\n8YnqElkvlLv6J/CnRd+ydd9xOrWsWadOm7opkPMY+wCZqpoFICJzgRF49ihKKVA6htEY2FO6QERG\nAlnAifIUJp6B4NuAq8uznqleTRqE88SN3fnVkK7UCxUbv78AY/q156XPMpm+NItnbrXpTUzwC+Sc\nwbbALq/XOU6bt6nAGBHJwbP3cT+AiEQBjwBP+tmuAh+LSIaITPKzPAXYp6pbvdoSRWSNiHwuIin+\nihWRSSKSLiLpubm5AXw8UxnCw0IsPC5Qs6hwfti7Hf9as4f9x06XvYIxLgskQPz9VfA9cDIamKmq\nccAwYLaIhOAJjmmqmudnGwNUtRcwFLhPRFL9bHOO1+u9QLyq9gR+CbwlItE+66CqaaqarKrJsbHl\nO0PIGLeNH5hIYUkJs5btcLsUY8oUSIDkAO28XsfhNUTlGA/MA1DVZUAkEAP0BZ5xDpg/ADwmIpOd\nfnucx/3Au3iGygAQkTDgZuDt0jZVzVfVg87zDGAb0DnAz2lMjZAQE8X13VvxxvJsTnhd/2JMMAok\nQFYBnUQkUUTCgVHAfJ8+2cBgABHphidAclU1RVUTVDUBeA54WlVfEJEoEWnk9I8CrgM2eG3vGuAb\nVc0pbRCRWOeAPiKSBHTCc2zFmFplYmoSR08V8vf0XWV3NsZFZQaIqhYBk4GFeE6jnaeqG0XkKREZ\n7nR7EJgoIuvwDDuN1fOfH9wS+MLpvxL4QFU/8lo+irOHrwBSgfXOOu8AP1bVQ2V/RGNqlt7tm9K7\nfVNe+XI7RcUlZa9gjEvsQkJjgtDCjd9x7+wMXryjFzdc2trtckwdE+h1IDZzmzFB6JpuLUmMiSJt\nyTZq8z/yTM1mAWJMEAoNEcYPTGRdzlFW7TjsdjnG+GUBYkyQuqVXHM2iwklbYueKmOBkAWJMkKof\nHsqd/drzn8372Jbr71IqY9xlAWJMELuzf3siwkKYsdT2QkzwsQAxJojFNIzglt5x/GP17rNmODYm\nGFiAGBPkxg9MpLC4hNnLdrhdijFnsQAxJsh1iG3INd1a8vrynZwqKHa7HGPOsAAxpgaYlJrEkZOF\nvJNh05uY4GEBYkwNkNy+KT3aNWHGF9spLrELC01wsAAxpgYQEe5NTWLnwZMs2vSd2+UYA1iAGFNj\nXHdRK+KbNbALC03QsAAxpoYIDREmpCSyOvsIGTttImrjPgsQY2qQW3vH0aRBPdsLMUHBAsSYGqRB\neBh39mvPx5v2sf3ACbfLMXWcBYgxNcxd/ROoF2LTmxj3BRQgIjJERLaISKaITPGzPF5EFovIGhFZ\nLyLD/CzPE5GHvNp2iMjXIrJWRNK92qeKyG6nfa33tkTkUaeGLSJyfcU+sjE1W2yjCG7u1ZZ3MnI4\nmGfTmxj3lBkgzn3IXwSGAt2B0SLS3afbE3huddsTz+1oX/JZPg340M/mr1LVHn7ufDXNae+hqguc\nOro7274IGAK8VHqPdGPqmgkpieQXlTB7+U63SzF1WCB7IH2ATFXNUtUCYC4wwqePAtHO88bAntIF\nIjISyAI2XmCtI4C5qpqvqtuBTKc2Y+qcji0aMbhrC15ftpPThTa9iXFHIAHSFvCePyHHafM2FRgj\nIjnAAuB+ABGJAh4BnvSzXQU+FpEMEZnks2yyMxT2qog0LUcdiMgkEUkXkfTc3NwAPp4xNdPE1CQO\nnSjgH6tz3C7F1FGBBIj4afOdS2E0MFNV44BhwGwRCcETHNNU1d/dcAaoai88Q2P3iUiq0/4y0AHo\nAewF/liOOlDVNFVNVtXk2NjYMj6aMTVX38RmXBbXmBlLt1Ni05sYFwQSIDlAO6/XcXgNUTnGA/MA\nVHUZEAnEAH2BZ0RkB/AA8JiITHb67XEe9wPv4gxHqeo+VS1W1RJgOt8PUwVShzF1hogwMTWJ7QdO\n8J/N+9wux9RBgQTIKqCTiCSKSDieA9nzffpkA4MBRKQbngDJVdUUVU1Q1QTgOeBpVX1BRKJEpJHT\nPwq4DtjgvG7ttd0flLY77zlKRCJEJBHoBKws9yc2phYZclEr4prWZ7qd0mtcUGaAqGoRMBlYCGzG\nc7bVRhF5SkSGO90eBCaKyDpgDjBWVc+3T90S+MLpvxL4QFU/cpY945zeux64CviFU8dGPHs5m4CP\ngPtU1Y4emjotLDSE8QMTWbXjMKuzD7tdjqlj5Px/52u25ORkTU9PL7ujMTXYifwi+v/uEwZ2iuGl\nH/V2uxxTC4hIhp/LK/6LXYluTA0XFRHGmH7t+WjDd+w8aNObmOpjAWJMLTD2igRCQ4RXvtjudimm\nDrEAMaYWaBEdycgebZmXvovDJwrcLsfUERYgxtQSE1OTOF1Ywhs2vYmpJhYgxtQSnVs2YlCXWGYt\n22HTm5hqYQFiTC0yKSWJA3kF/GvNbrdLMXWABYgxtUj/Ds25uG0005dm2fQmpspZgBhTi4gIE1OS\n2JZ7gsVb9rtdjqnlLECMqWWGXdKatk3q233TTZWzADGmlqkXGsK4AQms2H6IdbuOuF2OqcUsQIyp\nhUb1iadRZJhNsmiqlAWIMbVQw4gw7ugbz4Kv97Lr0Em3yzG1lAWIMbXUuCsSCRGb3sRUHQsQY2qp\nVo0jGd6jDfPSd3HkpE1vYiqfBYgxtdjElCROFhTz5opst0sxtZAFiDG1WLfW0aR0imHmVzvIL7Lp\nTUzlCihARGSIiGwRkUwRmeJnebyILBaRNSKyXkSG+VmeJyIPebXtcO48uFZE0r3a/yAi3zjbeVdE\nmjjtCSJyyum/VkT+WvGPbUzdMSk1idzj+fx77R63SzG1TJkBIiKhwIvAUKA7MFpEuvt0ewLPrW57\n4rln+ks+y6cBH/rZ/FWq2sPnzleLgItV9VLgW+BRr2XbnP49VPXHZdVujIGBHWPo1jqa6UuyqM13\nIDXVL5A9kD5ApqpmqWoBMBcY4dNHgWjneWPgzD91RGQkkAVsDKQgVf3YuQ87wHIgLpD1jDH+iQiT\nUhPZuj+Pz77NdbscU4sEEiBtgV1er3OcNm9TgTEikgMsAO4HEJEo4BHgST/bVeBjEckQkUnneO97\nOHvPJdEZJvtcRFICqN0YA9x4aRtaRUcy3aY3MZUokAARP22++8GjgZmqGgcMA2aLSAie4Jimqnl+\ntjFAVXvhGRq7T0RSz3pTkceBIuBNp2kvEO8Mk/0SeEtEovEhIpNEJF1E0nNz7V9bxoBnepN7Bibw\n1baDbNh91O1yTC0RSIDkAO28XsfhNUTlGA/MA1DVZUAkEAP0BZ4RkR3AA8BjIjLZ6bfHedwPvItn\nqAwAEbkbuBH4kTqDtqqar6oHnecZwDags2+xqpqmqsmqmhwbGxvAxzOmbhjVJ56GETa9iak8gQTI\nKqCTiCSKSDieg+TzffpkA4MBRKQbngDJVdUUVU1Q1QTgOeBpVX1BRKJEpJHTPwq4DtjgvB6CZ9hr\nuKqemYNBRGKdA/qISBLQCc+xFWNMAKIj6zG6TzveX7+XnMM2vYm5cGUGiHNAezKwENiM52yrjSLy\nlIgMd7o9CEwUkXXAHGCsnv90j5bAF07/lcAHqvqRs+wFoBGwyOd03VRgvbPOO8CPVfVQuT6tMXXc\nuAGJCPDalzvcLsXUAlKbT+tLTk7W9PT0sjsaU4c8MHcNizbt46tHB9O4fj23yzFBSEQyfC6v8Muu\nRDemjpmQksSJgmLmrLTpTcyFsQAxpo65uG1jBnRszmtfbqegqMTtckwNZgFiTB00MSWJfcfyeW+d\nTW9iKs4CxJg66MrOsXRp2YjpS216E1NxFiDG1EEiwsTUJL757jhLtx5wuxxTQ1mAGFNHDb+sDS2j\nI+zCQlNhFiDG1FHhYSGMvSKRpVsPsGnPMbfLMTWQBYgxddgdfeOJCg9lhu2FmAqwADGmDmtcvx63\nXx7P/HV72HPklNvlmBrGAsSYOm7cgAQUmPnVDrdLMTWMBYgxdVy7Zg0Ydklr3lqRzbHThW6XY2oQ\nCxBjDBNTEsnLL+LtlbvK7myMwwLEGMOlcU3ol9SMV7/cTmGxTW9iAmMBYowBYFJqEnuPnuaD9Xvd\nLsXUEBYgxhgABnVuQacWDUlbYtObmMBYgBhjAAgJESamJLFp7zG+2nbQ7XJMDRBQgIjIEBHZIiKZ\nIjLFz/J4EVksImtEZL2IDPOzPE9EHvJq2yEiXzt3HUz3am8mIotEZKvz2NRpFxH5s1PDehHpVfGP\nbYzxZ0TPNsQ0jCBtiV1YaMpWZoA49yF/ERgKdAdGi0h3n25P4LnVbU8890x/yWf5NOBDP5u/SlV7\n+Nz5agrwiap2Aj5xXuO8fyfnZxLwclm1G2PKJyIslHEDEvj821y2fHfc7XJMkAtkD6QPkKmqWapa\nAMwFRvj0USDaed4YOHOTAREZCWQBGwOsaQQwy3k+Cxjp1f66eiwHmohI6wC3aYwJ0I/6xlO/XqhN\nsmjKFEiAtAW8Tw7Pcdq8TQXGiEgOsAC4H0BEooBHgCf9bFeBj0UkQ0QmebW3VNW9AM5ji3LUYYy5\nQE0ahHP75e3499rdfHf0tNvlmCAWSICInzbfUzRGAzNVNQ4YBswWkRA8wTFNVfP8bGOAqvbCMzR1\nn4ikVkIdiMgkEUkXkfTc3NwyNmmM8eeeAYkUl6hNb2LOK5AAyQHaeb2Ow2uIyjEemAegqsuASCAG\n6As8IyI7gAeAx0RkstNvj/O4H3gXz1AZwL7SoSnncX856kBV01Q1WVWTY2NjA/h4xhhf8c0bMPTi\n1ry5Yid5+UVul2OCVCABsgroJCKJIhKO5yD5fJ8+2cBgABHphidAclU1RVUTVDUBeA54WlVfEJEo\nEWnk9I8CrgM2ONuaD9ztPL8b+LdX+13O2Vj9gKOlQ13GmMo3ISWR46eLeHuVTW9i/CszQFS1CJgM\nLAQ24znbaqOIPCUiw51uDwITRWQdMAcYq+e/Eqkl8IXTfyXwgap+5Cz7PXCtiGwFrnVeg+fYShaQ\nCUwHflqOz2mMKaee8U3pk9CMV7/YTpFNb1JjqCq/eX8T89KrPvilNl9xmpycrOnp6WV3NMb4tWjT\nPia+ns6fR/dk+GVt3C7HlKGkRPn1/A28sTybewYk8uubfK+4CIyIZPhcXuGXXYlujDmnwV1bkBQb\nRdqSbTa9SZArLlEe/efXvLE8m3tTk/ifG7tV+XtagBhjzql0epMNu4+xPOuQ2+WYcygqLuGhv6/j\n7fRd/OzqjkwZ2hURfyeuVi4LEGPMef2gZ1tiGobbhYVBqrC4hJ/PXcu7a3bz0HWd+eV1XaolPMAC\nxBhThsh6odzVP4FPv9nP1n02vUkwyS8q5qdvruaDr/fy+LBuTL66U7W+vwWIMaZMY/q1J7JeCDOW\nbne7FOM4XVjMvbMzWLRpH0+NuIiJqUnVXoMFiDGmTM2iwvlh73a8u2Y3+4/Z9CZuO1lQxIRZ6Xz+\nbS6/u/kS7uqf4EodFiDGmICMH5hIYUkJs5btcLuUOi0vv4ixr63iq20HePbWyxjdJ961WixAjDEB\nSYiJ4vrurXhjeTYnbHoTVxw9Vcidr6wgY+dhnh/Vk1t6x7lajwWIMSZgE1OTOHqqkL9Xw1XO5mxH\nThYwZsYKNuw+yot39OKmILiw0wLEGBOw3u2b0rt9U1750qY3qU4H8/IZlbacLfuO87c7ezPk4lZu\nlwRYgBhjymliShK7Dp1i4cZ9bpdSJ+w/dppRacvZcfAEM+5K5uquLd0u6QwLEGNMuVzbvSWJMTa9\nSXXYe/QUt6ctZ/eRU7w2tg+pnYPrFhUWIMaYcgkNEcYPTGRdzlFW7Tjsdjm11q5DJ7ntb8s4cDyf\n2eP70L9Dc7dL+i8WIMaYcrulVxzNosJJW2LTm1SFHQdOMCptOUdPFvLGhL70bt/M7ZL8sgAxxpRb\n/fBQ7uzXnv9s3se2XH93rDYVlbk/j9vTlnGyoIg5k/pxWbsmbpd0ThYgxpgKubN/eyLCQphhkyxW\nmi3fHWdU2jKKS2DupP5c1KaUzAwrAAAUeUlEQVSx2yWdV0ABIiJDRGSLiGSKyBQ/y+NFZLGIrBGR\n9SIyzM/yPBF5yKc91Fnnfa+2pSKy1vnZIyL/ctoHichRr2W/rthHNsZUhpiGEdzSO45/rN5N7vF8\nt8up8TbsPsqotGWEhghv39uPLq0auV1SmcoMEBEJBV4EhgLdgdEi4nubqyfw3Oq2J557pr/ks3wa\n8KGfzf8cz21yz3Duo95DVXsAy4B/ei1eWrpMVZ8qq3ZjTNUaPzCRwuISZi/b4XYpNdraXUe4Y/py\nGoSHMe/e/nSIbeh2SQEJZA+kD5CpqlmqWgDMBUb49FEg2nneGNhTukBERuK5l/lG7xVEJA64AZjh\n701FpBFwNfCvAGo0xrigQ2xDrunWkteX7+RUQbHb5dRI6TsOMWbGCpo0COfte/vRvnmU2yUFLJAA\naQt4z1uQ47R5mwqMEZEcYAFwP4CIRAGPAE/62e5zwK+Ac13O+gPgE1U95tXWX0TWiciHInJRALUb\nY6rYpNQkjpws5J0Mm96kvL7adoC7Xl1Ji0YRvH1vP+KaNnC7pHIJJED83drK9+qh0cBMVY0DhgGz\nRSQET3BMU9WzTtMQkRuB/aqacZ73HQ3M8Xq9GmivqpcBf+EceyYiMklE0kUkPTc393yfyxhTCZLb\nN6VHuybM+GI7xSV2YWGglnyby7jXVtG2SX3m3tuP1o3ru11SuQUSIDlAO6/XcXgNUTnGA/MAVHUZ\nEAnEAH2BZ0RkB/AA8JiITAYGAMOd9rnA1SLyRunGRKQ5nqGzD0rbVPVYaRCp6gKgnojE+Barqmmq\nmqyqybGxwXXVpjG1kYgwKTWJnQdPsmjTd26XUyN8snkfE2alkxTbkLmT+tGiUaTbJVVIIAGyCugk\nIokiEo7nIPl8nz7ZwGAAEemGJ0BynQPiCaqagGfI6mlVfUFVH1XVOKd9FPCpqo7x2t4PgfdV9cyd\na0SklTg3+hWRPk7tB8v/kY0xle36i1oR36yBXVgYgI827OXHb2TQtXUj5kzsS/OGEW6XVGFlBoiq\nFgGTgYV4zpiap6obReQpERnudHsQmCgi6/AMO43VC5skZxRnD18B3ApscN7jz8CoC3wPY0wlCQ0R\nJqQksjr7CBk7D7ldTtCav24P9721hkvaNuaNCX1p0iDc7ZIuiNTmv8HJycmanp7udhnG1AknC4q4\n4vef0jexGX+7M9ntcoLOOxk5/OqddVye0IxXxl5Ow4gwt0s6JxHJUNUy/yPalejGmErRIDyMO/u1\n5+NN+9h+4ITb5QSVt1Zk8/A767iiQwwzx/UJ6vAoDwsQY0yluat/AvVCbHoTbzO/3M5j737NoM6x\nzLg7mfrhoW6XVGksQIwxlSa2UQQ392rLOxk5HMyz6U3Slmxj6nubuK57S/56Z28i69We8AALEGNM\nJZuQkkh+UQmzl+90uxRX/eWTrTy94BtuuLQ1L/6oFxFhtSs8wALEGFPJOrZoxOCuLXh92U5OF9a9\n6U1UlT9+vIU/LvqWm3u25fnbe1AvtHb+qa2dn8oY46qJqUkcOlHAP1bnuF1KtVJVfv/hN/zl00xG\nXd6OP/zwMsJqaXiABYgxpgr0TWzGpXGNmbF0OyV1ZHoTVeXJ9zbxtyVZ3NW/PU//4BJCQ/zNBFV7\nWIAYYyqdiDAxJYntB07wn8373C6nypWUKI+9u4GZX+1gwsBEnhx+ESG1PDzAAsQYU0WGXtyKuKb1\nmV7LT+ktLlEefmc9c1Zmc99VHXj8hm44sy7VehYgxpgqERYawviBiazacZjV2YfdLqdKFBaX8MDb\na/nH6hx+eW1nHr6+a50JD7AAMcZUoduS2xEdGVYrLywsKCrh/rfW8N66PTwypCs/G9zJ7ZKqnQWI\nMabKREWEMaZfez7a8B07D9ae6U1OFxbz4zcy+Gjjd/z6xu78ZFAHt0tyhQWIMaZKjb0igdAQ4ZUv\ntrtdSqU4VVDMxNfT+fSb/fxm5MXcMzDR7ZJcYwFijKlSLaIjGdmjLfPSd3H4RIHb5VyQE/lF3DNz\nFV9kHuCZWy9lTL/2bpfkKgsQY0yVm5iaxOnCEt6owdObHD9dyN2vrmTljkM8d3sPbktuV/ZKtZwF\niDGmynVu2YhBXWKZtWxHjZze5OjJQsa8spK1u47wl9E9GdGjrdslBQULEGNMtZiUksSBvAL+tWa3\n26WUy6ETBdwxYzmb9xzjr2N6M+yS1m6XFDQCChARGSIiW0QkU0Sm+FkeLyKLRWSNiKwXkWF+lueJ\nyEM+7aHOOu97tc0Uke0istb56eG0i4j82alhvYj0qthHNsa4oX+H5lzUJprpS7NqzPQmucfzGZ22\nnMz9eaTd1Ztrurd0u6SgUmaAiEgo8CIwFOgOjBaR7j7dnsBzr/SeeO5n/pLP8mnAh342/3M891n3\n9bCq9nB+1jptQ4FOzs8k4OWyajfGBA8RYVJqEttyT7B4y363yynTd0dPc3vaMrIPneTVsZczqEsL\nt0sKOoHsgfQBMlU1S1ULgLnACJ8+CkQ7zxsDe0oXiMhIIAvY6L2CiMQBNwAzAqx1BPC6eiwHmoiI\n7UsaU4MMu6Q1bZvUJ21JcF9YuPvIKW5PW8a+o6eZdU8fBnSMcbukoBRIgLQFdnm9znHavE0FxohI\nDrAAuB9ARKKAR4An/Wz3OeBXQImfZb91hqmmiUhEOepARCaJSLqIpOfm5pb12Ywx1aheaAjjBiSw\nYvsh1u064nY5fmUfPMltf13GoRMFzJ7Qlz6JzdwuKWgFEiD+JnbxHcAcDcxU1ThgGDBbRELwBMc0\nVc07a4MiNwL7VTXDz7YfBboClwPN8ARQoHWgqmmqmqyqybGxsef5WMYYN4zqE0+jyLCgnGQxKzeP\n29OWcaKgiLcm9KNXfFO3SwpqYQH0yQG8T3iOw2uIyjEeGAKgqstEJBKIAfoCt4rIM0AToERETuPZ\ncxjuHGyPBKJF5A1VHaOqe51t5ovIa0DpgfdA6jDGBLmGEWHc0Tee6Uuy2HXoJO2aNXC7JAC27jvO\nHTNWUFKizJnYj26to8teqY4LZA9kFdBJRBJFJBzPQfL5Pn2ygcEAItINTyjkqmqKqiaoagKeIaun\nVfUFVX1UVeOc9lHAp6o6xlm/tfMowEhgg/Me84G7nLOx+gFHvcLGGFODjLsikRAJnulNNu05xu1p\nyxHg7XstPAJVZoCoahEwGViI54ypeaq6UUSeEpHhTrcHgYkisg6YA4xV1Yqep/emiHwNfI1nL+Y3\nTvsCPAfjM4HpwE8ruH1jjMtaNY5keI82zEvfxZGT7k5vsj7nCKOnLyciLIS37+1PxxaNXK2nJpGK\n/50PfsnJyZqenu52GcYYPzbvPcbQ55fy8PVduO+qjq7UkLHzMGNfXUnjBvWYM7Ff0AynuU1EMlQ1\nuax+diW6McYV3VpHk9Iphplf7SC/qPqnN1mRdZC7XllB84bhzLu3v4VHBViAGGNcMyk1idzj+fx7\nbfWeD/PF1gPc/dpKWjWO5O17+9OmSf1qff/awgLEGOOagR1j6NqqEdOXZFFdw+mLv9nPPbNWkdA8\nirfv7U/L6Mhqed/ayALEGOOa0ulNtu7P47Nvq/7C3483fsek2el0btmQORP7EdMwouyVzDlZgBhj\nXHXTZW1oFR3J9Cqe3uSD9Xv56ZuruahNY96c0I+mUeFV+n51gQWIMcZV9UJDuGdgAl9tO8iG3Uer\n5D3eXZPD/XNW0zO+CbPH96Fx/XpV8j51jQWIMcZ1o/rE0zCiaqY3mbdqF7+ct45+Sc2ZdU8fGkVa\neFQWCxBjjOuiI+sxuk873l+/l91HTlXadmcv38mv/rGe1E6xvDr2chqEBzJ7kwmUBYgxJiiMG5CI\nAK9W0vQmM5Zm8T//2sA13VqQdldvIuuFVsp2zfcsQIwxQaFNk/rceGlr5q7M5uipwgva1kufZfKb\nDzYz9OJWvPSj3kSEWXhUBQsQY0zQmJCSxImCYuaszK7Q+qrKtEXf8sxHWxjRow1/Gd2T8DD7M1dV\n7Js1xgSNi9s2ZkDH5rz25XYKivzda+7cVJVnFm7h+U+2cmvvOP50Ww/CQu1PXFWyb9cYE1QmpiSx\n71g+760LfHoTVeU3H2zm5c+2cUffeJ655VJCQ/zdg85UJgsQY0xQubJzLF1aNmL60sCmNykpUf7n\n3xt45YvtjBuQwG9HXkyIhUe1sAAxxgQVEWFCSiLffHecpVsPnLdvcYny6D+/5o3l2dx7ZRK/vrE7\nnnvRmepgAWKMCTojerSlZXTEeS8sLCou4aG/r+Pt9F38bHAnpgzpauFRzQIKEBEZIiJbRCRTRKb4\nWR4vIotFZI2IrHfude67PE9EHvJpD3XWed+r7U3nvTaIyKsiUs9pHyQiR0VkrfPz64p9ZGNMsAsP\nC2HsFYks3XqATXuO/dfywuISfj53Le+u2c3D13fhl9d2tvBwQZkBIiKhwIvAUKA7MFpEuvt0ewLP\nrW574rnH+Us+y6cBH/rZ/M/x3CbX25tAV+ASoD4wwWvZUlXt4fw8VVbtxpia646+8USFhzLDZy8k\nv6iYn765mg++3ssTN3Rz7W6GJrA9kD5ApqpmqWoBMBcY4dNHgdK70DcGzpw+ISIj8dzLfKP3CiIS\nB9wAzDhrQ6oL1AGsBOIC/zjGmNqicf163H55PPPX7WHvUc/0JqcLi7l3dgaLNu3jqREXMSElyeUq\n67ZAAqQtsMvrdY7T5m0qMEZEcoAFwP0AIhIFPAI86We7zwG/Avye7O0MXd0JfOTV3F9E1onIhyJy\n0TnWmyQi6SKSnptb9fcXMMZUnXEDElDgtS93cLKgiPGzVvH5t7n87uZLuKt/gtvl1XmBBIi/gUXf\nc+tGAzNVNQ4YBswWkRA8wTFNVfPO2qDIjcB+Vc04z/u+BCxR1aXO69VAe1W9DPgL8C9/K6lqmqom\nq2pybGxsWZ/NGBPE2jVrwLBLWvPWimzGvrqKZdsO8uytlzG6T7zbpRkgkKkpc4B2Xq/j8BqicowH\nhgCo6jIRiQRigL7ArSLyDNAEKBGR03j2YIY7B9sjgWgReUNVxwCIyP8CscC9pW+gqse8ni8QkZdE\nJEZVz3+enzGmRpuYksh76/aQkX2Y50f15KbL2rhdknEEEiCrgE4ikgjsxnOQ/A6fPtnAYGCmiHTD\nEwq5qppS2kFEpgJ5qvqC0/So0z4IeMgrPCYA1wODVbXEa/1WwD5VVRHpg2fv6WD5Pq4xpqa5NK4J\njw/rRudWjbiys40qBJMyA0RVi0RkMrAQCAVeVdWNIvIUkK6q84EHgeki8gs8w1tjNZBLSP37K7AT\nWOaclvdP54yrW4GfiEgRcAoYdQHvYYypQSam2sHyYCS1+W9wcnKypqenu12GMcbUKCKSoarJZfWz\nK9GNMcZUiAWIMcaYCrEAMcYYUyEWIMYYYyrEAsQYY0yFWIAYY4ypEAsQY4wxFVKrrwMRkVw8FyVW\nVAwQjFOlWF3lY3WVj9VVPrWxrvaqWuZl/7U6QC6UiKQHcjFNdbO6ysfqKh+rq3zqcl02hGWMMaZC\nLECMMcZUiAXI+aW5XcA5WF3lY3WVj9VVPnW2LjsGYowxpkJsD8QYY0yF1MkAEZEhIrJFRDJFZIqf\n5WNFJFdE1jo/E7yW3S0iW52fu4OormKv9vnVWZfT5zYR2SQiG0XkLa92176vMupy7fsSkWle7/2t\niBzxWubm79f56nLz+4oXkcUiskZE1jt3Mi1d9qiz3hYRuT4Y6hKRBBE55fV9/bWa62ovIp84NX0m\nInFeyyr390tV69QPnptibQOSgHBgHdDdp89Y4AU/6zYDspzHps7zpm7X5SzLc/H76gSsKf0ugBZB\n8n35rcvt78un//14btLm+vd1rrrc/r7wjOX/xHneHdjh9XwdEAEkOtsJDYK6EoANLn5ffwfudp5f\nDcyuqt+vurgH0gfIVNUsVS0A5gIjAlz3emCRqh5S1cPAIpx7wbtcV1UKpK6JwIvOd4Kq7nfa3f6+\nzlVXVSrvf8fRwBznudvf17nqqkqB1KVAtPO8MbDHeT4CmKuq+aq6Hch0tud2XVUpkLq6A584zxd7\nLa/036+6GCBtgV1er3OcNl+3OLuA74hIu3KuW911AUSKSLqILBeRkZVUU6B1dQY6i8iXzvsPKce6\nbtQF7n5fgGeoAc+/nD8t77rVXBe4+31NBcaISA6wAM/eUaDrulEXQKIztPW5iKRUUk2B1rUOuMV5\n/gOgkYg0D3DdcqmLASJ+2nxPRXsPSFDVS4H/ALPKsa4bdQHEq+eq0zuA50SkQzXWFYZnuGgQnn+5\nzhCRJgGu60Zd4O73VWoU8I6qFldg3fK6kLrA3e9rNDBTVeOAYcBsEQkJcF036tqL5/vqCfwSeEtE\noqkcgdT1EHCliKwBrgR2A0UBrlsudTFAcgDvf7nH4bPrqaoHVTXfeTkd6B3oui7VharucR6zgM+A\nntVVl9Pn36pa6AwlbMHzh9vV7+s8dbn9fZUaxdnDRG5/X+eqy+3vazwwz3n/ZUAknnme3P6+/Nbl\nDKkddNoz8Byz6FxddanqHlW92Qmwx522owF+pvKpigM9wfyD51+lWXh20UsPQl3k06e11/MfAMv1\n+4NQ2/EcgGrqPG8WBHU1BSKc5zHAVs5zgLQK6hoCzPJ6/11A8yD4vs5Vl6vfl9OvC7AD51qsYPj9\nOk9dbv9+fQiMdZ53w/NHT4CLOPsgehaVdxD9QuqKLa0Dz8Hu3dX8ex8DhDjPfws8VVW/Xxf8gWri\nD57dzW/x/MvgcaftKWC48/x3wEbnP85ioKvXuvfgOViXCYwLhrqAK4CvnfavgfHVXJcAfwI2Oe8/\nKki+L791uf19Oa+nAr/3s65r39e56nL7+8JzUPhL5/3XAtd5rfu4s94WYGgw1IXn+EPp/6ergZuq\nua5b8YT8t8AMnPCvit8vuxLdGGNMhdTFYyDGGGMqgQWIMcaYCrEAMcYYUyEWIMYYYyrEAsQYY0yF\nWIAYY4ypEAsQY4wxFWIBYowxpkL+P+yMih7uj7SeAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x104116310>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# plot CV误差曲线\n",
    "test_means = grid_search.cv_results_[ 'mean_test_score' ]\n",
    "test_stds = grid_search.cv_results_[ 'std_test_score' ]\n",
    "train_means = grid_search.cv_results_[ 'mean_train_score' ]\n",
    "train_stds = grid_search.cv_results_[ 'std_train_score' ]\n",
    "\n",
    "x_axis = colsample_bytree_s\n",
    "\n",
    "plt.plot(x_axis, test_means)\n",
    "#plt.errorbar(x_axis, -test_scores[:,i], yerr=test_stds[:,i] ,label = str(max_depths[i]) +' Test')\n",
    "#plt.errorbar(x_axis, -train_scores[:,i], yerr=train_stds[:,i] ,label = str(max_depths[i]) +' Train')\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "再调小一点"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fitting 5 folds for each of 3 candidates, totalling 15 fits\n",
      "[CV] colsample_bytree=0.3 ............................................\n",
      "[CV] colsample_bytree=0.3 ............................................\n",
      "[CV] colsample_bytree=0.3 ............................................\n",
      "[CV] colsample_bytree=0.3 ............................................\n",
      "[CV] ....... colsample_bytree=0.3, score=0.861468816098, total=   0.9s\n",
      "[CV] colsample_bytree=0.3 ............................................\n",
      "[CV] ....... colsample_bytree=0.3, score=0.833063511119, total=   0.9s\n",
      "[CV] colsample_bytree=0.4 ............................................\n",
      "[CV] ....... colsample_bytree=0.3, score=0.824688342886, total=   1.0s\n",
      "[CV] colsample_bytree=0.4 ............................................\n",
      "[CV] ......... colsample_bytree=0.3, score=0.8607024904, total=   1.0s\n",
      "[CV] colsample_bytree=0.4 ............................................\n",
      "[CV] ....... colsample_bytree=0.3, score=0.849136424933, total=   0.8s\n",
      "[CV] colsample_bytree=0.4 ............................................\n",
      "[CV] ....... colsample_bytree=0.4, score=0.861435659978, total=   0.9s\n",
      "[CV] ....... colsample_bytree=0.4, score=0.836595666838, total=   0.9s\n",
      "[CV] colsample_bytree=0.4 ............................................\n",
      "[CV] colsample_bytree=0.5 ............................................\n",
      "[CV] ....... colsample_bytree=0.4, score=0.834527594932, total=   1.2s\n",
      "[CV] colsample_bytree=0.5 ............................................\n",
      "[CV] ....... colsample_bytree=0.4, score=0.860291089263, total=   0.9s\n",
      "[CV] colsample_bytree=0.5 ............................................\n",
      "[CV] ....... colsample_bytree=0.5, score=0.861714971703, total=   0.9s\n",
      "[CV] ....... colsample_bytree=0.4, score=0.852223310925, total=   1.0s\n",
      "[CV] colsample_bytree=0.5 ............................................\n",
      "[CV] colsample_bytree=0.5 ............................................\n",
      "[CV] ....... colsample_bytree=0.5, score=0.836188303893, total=   1.1s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=4)]: Done  12 out of  15 | elapsed:    4.3s remaining:    1.1s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV] ....... colsample_bytree=0.5, score=0.831381810407, total=   0.9s\n",
      "[CV] ....... colsample_bytree=0.5, score=0.850134164523, total=   0.8s\n",
      "[CV] ....... colsample_bytree=0.5, score=0.851513115937, total=   0.8s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=4)]: Done  15 out of  15 | elapsed:    5.0s finished\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "GridSearchCV(cv=StratifiedKFold(n_splits=5, random_state=3, shuffle=True),\n",
       "       error_score='raise',\n",
       "       estimator=LGBMClassifier(boosting_type='goss',\n",
       "        categorical_feature=[0, 1, 3, 5, 6, 12, 15, 16, 17, 18, 19, 20],\n",
       "        colsample_bytree=1.0, is_unbalance=True, learning_rate=0.1,\n",
       "        max_bin=255, max_depth=6, min_child_samples=40, min_child_weight=5,\n",
       "        min_split_gain=0.0, n_estimators=38, n_jobs=4, num_leaves=70,\n",
       "        objective='binary', random_state=0, reg_alpha=0.0, reg_lambda=0.0,\n",
       "        silent=False, subsample=1.0, subsample_for_bin=50000,\n",
       "        subsample_freq=1),\n",
       "       fit_params=None, iid=True, n_jobs=4,\n",
       "       param_grid={'colsample_bytree': [0.3, 0.4, 0.5]},\n",
       "       pre_dispatch='2*n_jobs', refit=False, return_train_score='warn',\n",
       "       scoring='roc_auc', verbose=5)"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "params = {'boosting_type': 'goss',\n",
    "          'objective': 'binary',\n",
    "          'is_unbalance':True,\n",
    "          'categorical_feature': [0,1,3,5,6,12,15,16,17,18,19,20],\n",
    "          'n_jobs': 4,\n",
    "          'learning_rate': 0.1,\n",
    "          'n_estimators':n_estimators_1,\n",
    "          'num_leaves': 70,\n",
    "          'max_depth': 6,\n",
    "          'min_child_samples':40\n",
    "          #'colsample_bytree': 0.7,\n",
    "         }\n",
    "lg = LGBMClassifier(silent=False,  **params)\n",
    "\n",
    "colsample_bytree_s = [i/10.0 for i in range(3,6)]\n",
    "tuned_parameters = dict( colsample_bytree = colsample_bytree_s)\n",
    "\n",
    "grid_search = GridSearchCV(lg, n_jobs=4,  param_grid=tuned_parameters, cv = kfold, scoring=\"roc_auc\", verbose=5, refit = False)\n",
    "grid_search.fit(X_train , y_train)\n",
    "#grid_search.best_estimator_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.849014497953\n",
      "{'colsample_bytree': 0.4}\n"
     ]
    }
   ],
   "source": [
    "# examine the best model\n",
    "print(grid_search.best_score_)\n",
    "print(grid_search.best_params_)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### colsample_bytree=0.4"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 正则化参数lambda_l1(reg_alpha), lambda_l2(reg_lambda)感觉不用调了"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 减小学习率，调整n_estimators"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "('best n_estimators:', 524)\n",
      "('best cv score:', 0.84939448240588145)\n"
     ]
    }
   ],
   "source": [
    "params = {'boosting_type': 'goss',\n",
    "          'objective': 'binary',\n",
    "          'is_unbalance':True,\n",
    "          'categorical_feature': [0,1,3,5,6,12,15,16,17,18,19,20],\n",
    "          'n_jobs': 4,\n",
    "          'learning_rate': 0.01,\n",
    "          #'n_estimators':n_estimators_1,\n",
    "          'num_leaves': 70,\n",
    "          'max_depth': 6,\n",
    "          'min_child_samples':40,\n",
    "          'colsample_bytree': 0.4\n",
    "         }\n",
    "n_estimators_2 = get_n_estimators(params , X_train , y_train, early_stopping_rounds=50)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 用所有训练数据，采用最佳参数重新训练模型\n",
    "由于样本数目增多，模型复杂度稍微扩大一点？\n",
    "同时num_leaves增多5\n",
    "#min_child_samples按样本比例增加到15"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LGBMClassifier(boosting_type='goss',\n",
       "        categorical_feature=[0, 1, 3, 5, 6, 12, 15, 16, 17, 18, 19, 20],\n",
       "        colsample_bytree=0.4, is_unbalance=True, learning_rate=0.01,\n",
       "        max_bin=255, max_depth=6, min_child_samples=40, min_child_weight=5,\n",
       "        min_split_gain=0.0, n_estimators=10, n_jobs=4, num_leaves=75,\n",
       "        objective='binary', random_state=0, reg_alpha=0.0, reg_lambda=0.0,\n",
       "        silent=False, subsample=1.0, subsample_for_bin=50000,\n",
       "        subsample_freq=1)"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "params = {'boosting_type': 'goss',\n",
    "          'objective': 'binary',\n",
    "          'is_unbalance':True,\n",
    "          'categorical_feature': [0,1,3,5,6,12,15,16,17,18,19,20],\n",
    "          'n_jobs': 4,\n",
    "          'learning_rate': 0.01,\n",
    "          #'n_estimators':n_estimators_1,\n",
    "          'num_leaves': 75,\n",
    "          'max_depth': 6,\n",
    "          'min_child_samples':40,\n",
    "          'colsample_bytree': 0.4\n",
    "         }\n",
    "\n",
    "lg = LGBMClassifier(silent=False,  **params)\n",
    "lg.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 保存模型，用于后续测试"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import cPickle\n",
    "\n",
    "cPickle.dump(lg, open(\"HappyBank_LightGBM_.pkl\", 'wb'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 特征重要性"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "df = pd.DataFrame({\"columns\":list(feat_names), \"importance\":list(lg.feature_importances_.T)})\n",
    "df = df.sort_values(by=['importance'],ascending=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>columns</th>\n",
       "      <th>importance</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>Monthly_Income</td>\n",
       "      <td>82</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>City</td>\n",
       "      <td>79</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Loan_Amount_Submitted</td>\n",
       "      <td>49</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Loan_Amount_Applied</td>\n",
       "      <td>43</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>Age</td>\n",
       "      <td>40</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Interest_Rate</td>\n",
       "      <td>39</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>EMI_Loan_Submitted</td>\n",
       "      <td>37</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>Loan_Tenure_Applied</td>\n",
       "      <td>37</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>Processing_Fee</td>\n",
       "      <td>29</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>Var5</td>\n",
       "      <td>28</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Existing_EMI</td>\n",
       "      <td>24</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>Var1</td>\n",
       "      <td>17</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>Source</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Employer_Name</td>\n",
       "      <td>12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Filled_Form</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>Salary_Account</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Device_Type</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>Loan_Tenure_Submitted</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>Var4</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Gender</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>Var2</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>Mobile_Verified</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                  columns  importance\n",
       "13         Monthly_Income          82\n",
       "0                    City          79\n",
       "9   Loan_Amount_Submitted          49\n",
       "8     Loan_Amount_Applied          43\n",
       "21                    Age          40\n",
       "7           Interest_Rate          39\n",
       "2      EMI_Loan_Submitted          37\n",
       "10    Loan_Tenure_Applied          37\n",
       "14         Processing_Fee          29\n",
       "20                   Var5          28\n",
       "4            Existing_EMI          24\n",
       "17                   Var1          17\n",
       "16                 Source          16\n",
       "3           Employer_Name          12\n",
       "5             Filled_Form           9\n",
       "15         Salary_Account           8\n",
       "1             Device_Type           7\n",
       "11  Loan_Tenure_Submitted           7\n",
       "19                   Var4           6\n",
       "6                  Gender           4\n",
       "18                   Var2           2\n",
       "12        Mobile_Verified           1"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD8CAYAAABn919SAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAADxxJREFUeJzt3W+MHVd9xvHvQ0IEhFZJyNpyE1yH\nykqDKmHoKkqbCkFMaCgIuxKpiCpkVa7cF0ChUBXDG9qqlYzUEvqiQnIJZV9A/hCS2gJEsawgWqly\nWYeUJBjkkJpg7NoLJCUtVanh1xd3LIzZ9Z3dvde79+z3I0Uzc+6M70+TyeOTc+fMpKqQJE2+56x0\nAZKk0TDQJakRBrokNcJAl6RGGOiS1AgDXZIaYaBLUiMMdElqhIEuSY249GJ+2dVXX12bNm26mF8p\nSRPv8OHD36mqqWH7XdRA37RpE7OzsxfzKyVp4iX5Zp/9HHKRpEYY6JLUCANdkhphoEtSIwx0SWqE\ngS5JjTDQJakRBrokNcJAl6RGXNSZopJ+2qbdn+m977E9rx9jJWpBrx56kj9K8niSx5LcneR5Sa5L\ncijJ0ST3Jrls3MVKkhY2NNCTXAP8ITBdVb8CXAK8GfgAcGdVbQaeBnaOs1BJ0oX1HUO/FHh+kkuB\nFwAngVuA+7vPZ4Dtoy9PktTX0ECvqm8DfwU8xSDI/xM4DDxTVWe63Y4D14yrSEnScH2GXK4EtgHX\nAb8AXA68bp5da4HjdyWZTTI7Nze3nFolSRfQZ8jlNcC/V9VcVf0f8ADw68AV3RAMwLXAifkOrqq9\nVTVdVdNTU0Ofzy5JWqI+ty0+BdyU5AXA/wBbgVngIeBNwD3ADmDfuIoEb++SpGH6jKEfYvDj58PA\no90xe4H3AO9K8gTwIuCuMdYpSRqi18Siqno/8P7zmp8Ebhx5RZKkJXHqvyQ1wkCXpEYY6JLUCANd\nkhphoEtSIwx0SWqEgS5JjTDQJakRBrokNcJAl6RGGOiS1AgDXZIaYaBLUiMMdElqhIEuSY0w0CWp\nEX1eEn19kkfO+ef7Sd6Z5KokB5Ic7ZZXXoyCJUnz6/MKuq9X1Zaq2gL8KvAD4EFgN3CwqjYDB7tt\nSdIKWeyQy1bgG1X1TWAbMNO1zwDbR1mYJGlxFhvobwbu7tbXV9VJgG65bpSFSZIWp3egJ7kMeCPw\nycV8QZJdSWaTzM7NzS22PklST4vpob8OeLiqTnXbp5JsAOiWp+c7qKr2VtV0VU1PTU0tr1pJ0oIW\nE+h38JPhFoD9wI5ufQewb1RFSZIWr1egJ3kBcCvwwDnNe4BbkxztPtsz+vIkSX1d2menqvoB8KLz\n2r7L4K4XSdIq4ExRSWqEgS5JjTDQJakRBrokNcJAl6RGGOiS1AgDXZIaYaBLUiMMdElqhIEuSY0w\n0CWpEQa6JDXCQJekRhjoktQIA12SGmGgS1IjDHRJakTfV9BdkeT+JF9LciTJryW5KsmBJEe75ZXj\nLlaStLC+PfS/AT5XVb8MvAw4AuwGDlbVZuBgty1JWiFDAz3JzwOvBO4CqKofVtUzwDZgptttBtg+\nriIlScP16aG/BJgD/j7Jl5N8JMnlwPqqOgnQLdfNd3CSXUlmk8zOzc2NrHBJ0k/rE+iXAq8APlxV\nLwf+m0UMr1TV3qqarqrpqampJZYpSRqmT6AfB45X1aFu+34GAX8qyQaAbnl6PCVKkvoYGuhV9R/A\nt5Jc3zVtBb4K7Ad2dG07gH1jqVCS1MulPfd7O/DxJJcBTwK/x+Avg/uS7ASeAm4fT4mSpD56BXpV\nPQJMz/PR1tGWI0laKmeKSlIjDHRJaoSBLkmNMNAlqREGuiQ1wkCXpEYY6JLUCANdkhphoEtSIwx0\nSWqEgS5JjTDQJakRBrokNcJAl6RGGOiS1AgDXZIa0esFF0mOAc8CPwLOVNV0kquAe4FNwDHgd6rq\n6fGUKUkaZjE99FdX1ZaqOvvmot3AwaraDBzstiVJK2Q5Qy7bgJlufQbYvvxyJElL1TfQC/h8ksNJ\ndnVt66vqJEC3XDeOAiVJ/fQaQwdurqoTSdYBB5J8re8XdH8B7ALYuHHjEkqUJPXRq4deVSe65Wng\nQeBG4FSSDQDd8vQCx+6tqumqmp6amhpN1ZKknzG0h57kcuA5VfVst/5a4M+B/cAOYE+33DfOQrU2\nbdr9md77Htvz+jFWIq1+fYZc1gMPJjm7/yeq6nNJvgTcl2Qn8BRw+/jKlCQNMzTQq+pJ4GXztH8X\n2DqOoiRJi+dMUUlqhIEuSY0w0CWpEQa6JDXCQJekRhjoktQIA12SGmGgS1IjDHRJaoSBLkmNMNAl\nqREGuiQ1ou8LLqRl8TG40vjZQ5ekRhjoktQIA12SGtE70JNckuTLST7dbV+X5FCSo0nuTXLZ+MqU\nJA2zmB76O4Aj52x/ALizqjYDTwM7R1mYJGlxegV6kmuB1wMf6bYD3ALc3+0yA2wfR4GSpH769tA/\nBPwJ8ONu+0XAM1V1pts+Dlwz4tokSYsw9D70JG8ATlfV4SSvOts8z661wPG7gF0AGzduXGKZkrS6\nrMa5FX166DcDb0xyDLiHwVDLh4Arkpz9C+Fa4MR8B1fV3qqarqrpqampEZQsSZrP0B56Vb0XeC9A\n10P/46r63SSfBN7EIOR3APvGWKdWidXYK5E0sJz70N8DvCvJEwzG1O8aTUmSpKVY1LNcquoLwBe6\n9SeBG0dfkiRpKZwpKkmNMNAlqRE+PncB/vg32fz3p7XIHrokNcJAl6RGGOiS1AgDXZIaYaBLUiMM\ndElqhIEuSY0w0CWpEQa6JDXCQJekRhjoktQIA12SGmGgS1IjhgZ6kucl+dck/5bk8SR/1rVfl+RQ\nkqNJ7k1y2fjLlSQtpE8P/X+BW6rqZcAW4LYkNwEfAO6sqs3A08DO8ZUpSRpmaKDXwH91m8/t/ing\nFuD+rn0G2D6WCiVJvfQaQ09ySZJHgNPAAeAbwDNVdabb5ThwzXhKlCT10SvQq+pHVbUFuJbBi6Fv\nmG+3+Y5NsivJbJLZubm5pVcqSbqgRd3lUlXPAF8AbgKuSHL2FXbXAicWOGZvVU1X1fTU1NRyapUk\nXUCfu1ymklzRrT8feA1wBHgIeFO32w5g37iKlCQN1+cl0RuAmSSXMPgL4L6q+nSSrwL3JPkL4MvA\nXWOsU9I5fAm25jM00KvqK8DL52l/ksF4uiRpFejTQ5ekZrX0fztO/ZekRhjoktQIA12SGmGgS1Ij\nDHRJaoSBLkmNMNAlqREGuiQ1wkCXpEY4U3SVaGm2mqSVYQ9dkhphoEtSIwx0SWqEgS5JjTDQJakR\nBrokNaLPO0VfnOShJEeSPJ7kHV37VUkOJDnaLa8cf7mSpIX06aGfAd5dVTcANwFvTfJSYDdwsKo2\nAwe7bUnSChka6FV1sqoe7tafBY4A1wDbgJlutxlg+7iKlCQNt6gx9CSbGLww+hCwvqpOwiD0gXUL\nHLMryWyS2bm5ueVVK0laUO9AT/JC4FPAO6vq+32Pq6q9VTVdVdNTU1NLqVGS1EOvQE/yXAZh/vGq\neqBrPpVkQ/f5BuD0eEqUJPXR5y6XAHcBR6rqg+d8tB/Y0a3vAPaNvjxJUl99nrZ4M/AW4NEkj3Rt\n7wP2APcl2Qk8Bdw+nhIlSX0MDfSq+mcgC3y8dbTlSJKWypmiktQIA12SGuEbiySNxWLewgW+iWsU\n7KFLUiPsoUu6IHvak8MeuiQ1wkCXpEYY6JLUCANdkhphoEtSIwx0SWqEgS5JjTDQJakRBrokNcKZ\noiO2mFl1zqiTNEr20CWpEX1eQffRJKeTPHZO21VJDiQ52i2vHG+ZkqRh+vTQPwbcdl7bbuBgVW0G\nDnbbkqQVNDTQq+qLwPfOa94GzHTrM8D2EdclSVqkpY6hr6+qkwDdct3oSpIkLcXYfxRNsivJbJLZ\nubm5cX+dJK1ZSw30U0k2AHTL0wvtWFV7q2q6qqanpqaW+HWSpGGWeh/6fmAHsKdb7htZRVoU73uX\ndFaf2xbvBv4FuD7J8SQ7GQT5rUmOArd225KkFTS0h15Vdyzw0dYR1yJJWgan/ktrhC97bp9T/yWp\nEc330P3RUIvh9aJJZg9dkhphoEtSI5ofcpG0NjhcZg9dkpphoEtSIwx0SWqEgS5JjfBH0TXKH5Ck\n9thDl6RGGOiS1AgDXZIaYaBLUiP8UVTSquJjfpduWT30JLcl+XqSJ5LsHlVRkqTFW3IPPcklwN8y\neAXdceBLSfZX1VdHVZw0KbwNVKvBcnroNwJPVNWTVfVD4B5g22jKkiQt1nIC/RrgW+dsH+/aJEkr\nIFW1tAOT24HfrKrf77bfAtxYVW8/b79dwK5u83rg60svd15XA98Z8Z856Twn8/O8zM/zMr/VdF5+\nsaqmhu20nLtcjgMvPmf7WuDE+TtV1V5g7zK+54KSzFbV9Lj+/EnkOZmf52V+npf5TeJ5Wc6Qy5eA\nzUmuS3IZ8GZg/2jKkiQt1pJ76FV1JsnbgH8ELgE+WlWPj6wySdKiLGtiUVV9FvjsiGpZqrEN50ww\nz8n8PC/z87zMb+LOy5J/FJUkrS4+y0WSGjGxge5jB+aX5FiSR5M8kmR2petZKUk+muR0ksfOabsq\nyYEkR7vllStZ40pY4Lz8aZJvd9fMI0l+ayVrvNiSvDjJQ0mOJHk8yTu69om7XiYy0M957MDrgJcC\ndyR56cpWtaq8uqq2TNotVyP2MeC289p2AwerajNwsNteaz7Gz54XgDu7a2ZL99vYWnIGeHdV3QDc\nBLy1y5OJu14mMtDxsQMaoqq+CHzvvOZtwEy3PgNsv6hFrQILnJc1rapOVtXD3fqzwBEGs94n7nqZ\n1ED3sQMLK+DzSQ53s3T1E+ur6iQM/iMG1q1wPavJ25J8pRuSWfVDC+OSZBPwcuAQE3i9TGqgZ542\nb9cZuLmqXsFgOOqtSV650gVp1fsw8EvAFuAk8NcrW87KSPJC4FPAO6vq+ytdz1JMaqD3euzAWlRV\nJ7rlaeBBBsNTGjiVZANAtzy9wvWsClV1qqp+VFU/Bv6ONXjNJHkugzD/eFU90DVP3PUyqYHuYwfm\nkeTyJD93dh14LfDYhY9aU/YDO7r1HcC+Faxl1TgbWp3fZo1dM0kC3AUcqaoPnvPRxF0vEzuxqLu1\n6kP85LEDf7nCJa24JC9h0CuHwSzgT6zV85LkbuBVDJ6Ydwp4P/APwH3ARuAp4PaqWlM/EC5wXl7F\nYLilgGPAH5wdO14LkvwG8E/Ao8CPu+b3MRhHn6jrZWIDXZL00yZ1yEWSdB4DXZIaYaBLUiMMdElq\nhIEuSY0w0CWpEQa6JDXCQJekRvw/iQz45ha1KCkAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1a0c07ac90>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.bar(range(len(lg.feature_importances_)), lg.feature_importances_)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
